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value is now the goal of all negotiated Government procurements. One measure of contracting 
officer effectiveness is to examine protest decisions handed down by the General Accounting 
Office (GAO). This thesis examines all protest decisions of best value awards from 1997. The 
research reveals a sustainment (success) rate of 19.44 percent for 1997. GAO’s published 
sustainment rate for all protests is 12 percent for the same year. 

Best value sustainments (21 total) are first categorized in this thesis by agency 
improprieties in the evaluation of a tradeoff element (e.g. past performance, technical merit, 
cost/price, or labor qualifications) or improper pre- or post-award changes. The sustainments are 
then thoroughly analyzed to reveal pitfalls which contracting officers must avoid to preclude 
protest sustainment. 

These pitfalls are then incorporated into a final analysis where they are merged with the 
contracting process (acquisition planning, solicitation, source evaluation/selection, negotiation, 
award, contract administration) and examined for greater clarity. Examples of the common 
pitfalls which resulted in sustainment in 1997 are uncertainty of requirements, poorly crafted 
solicitations, failure to follow solicitations, failure to use all relevant facts, failure to evaluate total 
cost/price, improper cost/price realism analyses, pre-award solicitation changes without 
modification, failure to hold meaningful discussions when required, failure to support contract 


award with narrative, out-of-scope post award changes, and contract administrative improprieties. 
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I. INTRODUCTION 

A. BACKGROUND 

Best Value is the driving theme in the newly approved rewrite of Federal 
Acquisition Regulation (FAR) Part 15. The stated purpose of this was to infuse 
innovative techniques into the source selection process, simplify the process, and 
facilitate contract award under best value criteria. [Ref. 1: p. 51224] The task at hand is 
for Federal agency contracting officers to implement best value source selection into 
Government procurement in a manner consistent with Congress’ intentions. One method 
used by Congress to exert its right to influence and investigate Government acquisition is 
through the system of award protest. Congress’ investigative arm, the General 
Accounting Office (GAO) provides an arena for contract award protests and renders non- 
binding but influential decisions regarding source selection. This mechanism is the result 
of a statutory mandate under the Competition in Contracting Act (CICA) of 1984 tasking 
GAO to assess the reasonableness of source selection decisions and determine whether a 
procurement has been conducted in accordance with all applicable statutes and 
regulations [Ref. 2]. The overarching goal of CICA is full and open competition, and this 
Act has dominated Government contracting for over a decade. GAO protest decisions are 
a key indicator of Congress’ intent in the domain of Government procurement. Federal 
agencies of the Executive Branch should look to such decisions for messages from the 


Legislative Branch. 


During a protest, GAO requires an agency to provide certain information 
consistent with FAR documentation requirements so that it may be established whether 
the evaluation of proposals and the source selection decision were reasonable. The FAR 
has always required agencies to document the relative strengths, deficiencies, significant 
weakness, and risks supporting the proposal evaluation and source selection. However, 
the FAR Part 15 rewrite recognizes the subjective nature inherent in best value 
contracting. Contracting officers must now balance tradeoffs in an effort to obtain the 
best value for the Government. Tradeoffs are appropriate when it may be in the best 
interest of the Government to consider award to an offeror other than the lowest priced 
offeror or the highest technically rated offeror. As a result, the FAR Part 15 rewrite 
requires that in addition to common quantitative measures and matrices used in proposal 
evaluation, the agency shall document rationale for any business judgments and tradeoffs 
including supporting narrative [Ref. 1: 15.305, 15.308]. GAO has recently upheld 
protests because adjectival ratings needed to be accompanied by a reasoned explanation 
for cost/non-cost tradeoffs. In addition, agency contracting officers now face increased 
GAO scrutiny of their judgment of what constitutes best value in addition to the typical 
review of adjectival ratings, matrices, technical appraisals, and cost evaluation. 

B. OBJECTIVE 
The goal of this research is to ascertain whether or not best value contracting is 


more susceptible to sustainable protests. The thesis focuses on contracting officer 


judgment in source selection and evaluates the difficulties regarding contract award 
through best value determination. This document outlines the genesis and background of 
best value, describes laws and regulations that apply, and researches the 146 Comptroller 
General protest decisions from 1997. Lessons from the sustained protests of 1997 have 
been developed to assist contracting officers to effectively implement best value in source 
selection. 
C. THE RESEARCH QUESTION 
The primary research question is: “To what extent does the exercise of Federal 
agency contracting officer judgment, in best value source selection, result in a higher rate 
of GAO protest sustainment than other methods of source selection?” The subsidiary 
research questions are: 
e What is best value, the genesis of the concept, the evolution of this type of 
contracting, current implementation, and the regulations that apply? 
e Can the types of award protests and subsequent sustainments be categorized? 
If so, what are the categories? 
e Are there measures that a contracting officer can take to preclude protests 
regarding best value awards? 


e How may contracting officers be better trained regarding best value awards? 


D. SCOPE, METHODOLOGY, LIMITATIONS, AND ASSUMPTIONS 


This study focuses on the current nature and pitfalls inherent to best value 


contracting. The primary concentration of study consists of the 146 decisions rendered 


by the Comptroller General in response to protests of contract award under best value 


criteria. Other areas of study include a literature search of Government regulations, 


magazine articles, commentary, and other library information resources. The 


methodology consists of the following: 


l. 


u) 


A brief examination of all best value protests to identify sustainments and 
basic statistics. 

A thorough examination of the sustained protests. 

Categorization of the sustained protests. 

Analysis of the sustained protests. 

Generation of recommendations and conclusions regarding the sustained 


protests. 


This research is limited to best value protests that allege Federal agency 


improprieties in the determination of a best value award and/or Federal agency actions 


that undermine an otherwise proper best value contract award. The primary assumption 


in this study 1s that the reader is familiar with the basic rudiments, types, and processes of 


Government contracting. 


E. ORGANIZATION OF THE THESIS 

The following chapters in this thesis document the significance of best value and 
examine the protests involving best value contracting in 1997. Chapter II is a review of 
best value contracting including the genesis, evolution, and current state of this type of 
acquisition. Chapter III 1s an overall study of all best value protests in 1997. The study 
includes a review of the protest system, overall number of protests, and the overall 
success of protests in general throughout 1997. This chapter also categorizes the 
recurring reasons for protest. Chapters IV, V, VI, and VII review and analyze the best 
value protests that were sustained due to Federal agency improprieties in the evaluation 
of past performance, technical merit, cost/price, and labor qualifications respectively. 
Chapter VIII reviews and analyzes protests that were sustained due to improper Federal 
agency pre- and post-award changes. Chapter IX provides a final analysis and 


conclusion. 





IL REVIEW OF BEST VALUE CONTRACTING 

A. INTRODUCTION 

This chapter provides background into the concept of best value. The purpose of 
this chapter is to provide the reader with a firm grasp of the concept from its inception to 
its present state. Thorough understanding of the best value concept provides the reader 
with a firm foundation of knowledge to make better use of the material discussed in the 
remaining chapters of the thesis. This chapter addresses best value in the following 
manner. First, important definitions of best value terms are outlined. Second, the genesis 
of the best value concept is discussed. Third, the evolution of best value is detailed. 
Finally, a review of the current state of best value is conducted. 
B. DEFINITIONS 

The following definitions are derived from the Federal Acquisition Regulation 
(FAR) and are important to understand fully the nature and current state of best value 
contracting in the Federal Government. 

1. Best Value 

The expected outcome of an acquisition that, in the Government’s estimation, 
provides the greatest overall benefit in response to the requirement. [Ref. 1: 2.101] 

2. Best Value Continuum 

An agency can obtain best value in negotiated acquisitions by using any one ora 


combination of source selection approaches. In different types of acquisitions, the 


Ze Best Value Continuum 

An agency can obtain best value in negotiated acquisitions by using any one or a 
combination of source selection approaches. In different types of acquisitions, the 
relative importance of cost or price may vary. The choosing of the a selection 
approach will normally relate to the relative importance between cost and non-cost 
factors. [Ref. 1: 15.101] 

3. Tradeoff Process 

In negotiated procurement, the approach most commonly used to achieve a best 
value selection is a tradeoff process where dollars are traded for incremental value and 
cost is only one factor, not the controlling factor. A tradeoff process is appropriate when 
it may be in the best interest of the Government to consider award to other than the 
lowest priced offeror or other than the highest technically rated offeror. [Ref. 1: 15.101-1] 

4. Lowest Price-Technically Acceptable 

The lowest price-technically acceptable process is appropriate when best value is 
expected to result from selection of the technically acceptable proposal with the lowest 
evaluated price. [Ref. 1: 15.101-2] 
C GENESIS OF BEST VALUE 

There is no definitive starting point to the concept of best value. It is clear that 
businesses have historically sought to purchase goods and services advantageous to their 


pursuit of profit. The Government is not motivated by profit; however, increased 


emphasis has been placed on the efficiency of the acquisition process over the past 15 
years. The genesis of best value contracting in Government acquisition can be explored 
through study of the Government’s efforts to reform the acquisition process and adopt 
commercial practices. [Ref. 7] 

1. Concept Origin 

During the mid-1980’s a series of acquisition reform measures began that affect 
every aspect of Government contracting to this day. Most of these measures addressed 
best value either directly or indirectly. The first move toward the current state of best 
value occurred through the Government’s focus on the use of commercial practices. The 
1986 Packard Commission Final Report concluded that the use of commercial practices 
in the DoD acquisition process had the potential for saving vast amounts of Federal 
funds. In conjunction, the 1986 Defense Science Board stated that program managers 
should have discretionary authority to use commercial practices and products when 
appropriate. Commercial practices are “techniques, methods, customs, processes, rules, 
guides, and standards, normally used by businesses, but either applied differently or not 
used by the Federal Government.” [Ref. 4: p. 1-2] One such practice is best value. In the 
commercial sector, buyers use competition to their advantage, getting the maximum value 
at an affordable price. The objective is not necessarily lowest price or maximum 
performance, but a balance between the two. In 1989, DoD officially recognized the best 


value concept in a study entitled Best Value Evaluation Process. The Government had no 


set definition for best value at the time because the best value constantly changed 
according to the requirement of each individual purchase. Best value was generally 
accepted to mean buying on other than a price only basis. [Ref. 4: p. 1-1, 3-2] 

2 Competition In Contracting Act (CICA) Clarification 

A 1991 clarıfıcation by Congress regarding the 1984 Competition in Contracting 
Act (CICA) prompted the Government to focus on best value. CICA’s drive for free and 
open competition was sparked by the assumption that competition would result in lower 
acquisition costs for the Government as well as expanding contracting opportunities to 
every level of business. Initially there was confusion between GAO and DoD regarding 
the legislation because many in DoD improperly interpreted the Act as defining lowest 
cost to mean lowest price rather than lowest long-term costs. In 1991, the House Armed 
Services Committee Report for the Defense Authorization Act issued a clarification 
stating that Congress had attempted to make clear that lowest overall cost was not limited 
to price and price-related factors. Lowest overall cost was intended to be the outlay or 
expenditure that the Government would make over the life of a product. The committee 
went on to state that cost also encompasses technical factors such as quality, design, 
technical capability, management capability, past performance, and cost discipline to the 
extent these factors can be translated into a monetary context, and offerors can be given a 
clear indication in the solicitation how the factors would be quantified. This new 


interpretation of CICA led to the introduction of best value. [Ref. 5: p. 59-60] 
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D. EVOLUTION OF BEST VALUE CONTRACTING 

le Federal Acquisition Streamlining Act 

The Federal Acquisition Streamlining Act (FASA) of 1994 enacted sweeping 
acquisition reform over a wide spectrum of issues. The relationship between FASA and 
best value was indirect; however, it is important to note that this Act drove many 
elements of the FAR Part 15 rewrite and its specific emphasis on best value contracting. 
A key concept in FASA was a movement of Government contracting to commercial 
methods. This requirement of commercial contracting methods was the vital element 
leading to the Government’s future use of best value acquisition. 

FASA required that agencies clearly establish the relative importance assigned to 
evaluation factors and sub-factors. FASA specifically addressed the quality of the 
product or services to be provided including technical capability, management capability, 
prior experience, and past performance of the offeror. Cost was carefully addressed as 
only one of many evaluation factors as FASA required all combined evaluation factors 
other than cost or price be disclosed as significantly more important, approximately 
equal, or significantly less important than cost or price. This portion of the Act provided 
the groundwork for the tradeoff process which is essential to best value contracting. 
Through its recognition of other evaluation factors besides cost, FASA allowed 
Government agencies to venture into the spectrum of options now available through the 


use of tradeoffs. [Ref. 6: Sec. 1011] 
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2 Clinton-Gore Administration 

Best value contracting received high-level endorsement in the first term of the 
Clinton-Gore Administration. Vice President Gore in the Report of the National 
Performance Review: Creating a Government that Works Better & Costs Less recognized 
the need for deregulation in the acquisition process. The report maintained that 
acquisition regulations should: shift from rigid rules to guiding principles; promote 
decision making at the lowest possible level; end unnecessary regulatory requirements; 
foster competitiveness and commercial practices; and shift to a new emphasis on 
choosing “best value” products. In early 1996, the FAR Council pursued this shift to best 
value contracting by tasking an ad hoc interagency committee to rewrite FAR Part 15. 
[Ref. 1: p. 51224] 

3. Federal Acquisition Regulation (FAR) Part 15 Rewrite 

FAR Part 15 is one of the most important and significant parts of the Federal 
Acquisition Regulation because it addresses contracting by negotiation and competitive 
range determination. Within the Government, the majority of contracting expenditures are 
accomplished using Part 15 procedures. The Part 15 rewrite is a normal product of the 
continuous improvement process employed for maintenance of the FAR, and it conforms 
with the general reform philosophy espoused by the Clinton-Gore Administration. FAR 
Part 15 was rewritten for reasons other than the direction of the Executive Branch. 


Congress, through FASA in 1994 and the Clinger-Cohen Act in 1996 dictated so many 
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fundamental changes in Government acquisition that the rewrite became an obvious 
necessity. In 1995, the Department of Defense (DoD) conducted a survey of the defense 
industry, Military Departments, and defense agencies to ascertain which parts of the FAR 
were most in need of revision. The responses indicated a general consensus that Part 15 
was one of the parts that would benefit most from such an effort. In addition, a FAR 
Improvement Study conducted by the General Services Administration (GSA) indicated 
that subparts 15.6 (Source Selection) and 15.8 (Price Negotiation) were the most difficult 
parts of the FAR to use. The FAR Part 15 rewrite took effect on 10 October 1997. [Ref. 
09951225] 
E. CURRENT STATE OF BEST VALUE 

The current state of best value contracting from the perspective of the 
Government is contained within the FAR Part 15 rewrite. In sum, best value is the 
expected outcome of any acquisition that ensures the customer’s needs are met in the 
most effective, economical, and timely manner. It is the result of the combination of: the 
unique circumstances of each acquisition; the acquisition strategy; choice of contracting 
method; and the award decision. Best value is the goal of sealed bidding, simplified 
acquisition, commercial item acquisition, negotiated acquisition, and any other 
specialized acquisition method or combination of methods. Through the best value 
continuum the Government always seeks to obtain the best value in negotiated 


acquisitions using any one or a combination of source selection approaches, and that 
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acquisition should be tailored to the requirement. At one end of this continuum is the 
lowest priced-technically acceptable strategy and at the other end 1s a process by which 
elements of a proposed solution can be traded off against each other to determine the 
solution that provides the Government with the overall best value. All such tradeoffs 
must be conducted according to the source selection factors and sub-factors identified in 
the solicitation. [Ref. 3: p. 1] 
F, CONCLUSION 

The importance of best value cannot be overstated. The concept has been 
transformed from an undefined, nebulous entity to the mandated goal of all Government 
procurements. Best value is nothing new to the corporate sector; however, the 
Government is a relatively new participant in the world of best value. Government 
contracting officers must now employ this form of contracting without the benefit of 
significant experience to guide the way. The intent of this thesis is to counter the 
Government contracting officers’ lack of best value experience with a thorough review 
and analysis of recent protests of best value contract awards. Chapter III will review all 


protests of contract awards under best value criteria. 
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MI. OVERALL REVIEW OF FISCAL YEAR 1997 BEST VALUE PROTESTS 

A. INTRODUCTION 

This chapter provides an overall view of all best value protests for which the 
Comptroller General issued decisions in 1997. This chapter is beneficial because it gives 
the reader a firm understanding why contractors seeking Government business protested 
best value contract awards in 1997. First, a review of the protest process is conducted. 
Second, the methodology used to define a best value protest is outlined. Third, the 
protest sustainments are categorized. Finally, statistics are provided for each sustainment 
category. 
B. PROTEST PROCESS 

1. Purpose 

In deciding bid protests, GAO considers whether Federal agencies have 
complied with statutes and regulations controlling Government procurements. As stated 
in Chapter I, Congress exerts its right to influence and investigate Government 
acquisition through the system of award protest. Congress’ investigative arm, GAO 
provides a venue for contract award protest and renders non-binding but influential 


decisions regarding source selection. [Ref. 2] 


2: The Process 

The bid protest process begins with an unsuccessful offeror filing a written protest 
at GAO. Unless the protest is dismissed due to procedural or substantive defects (e.g., 
the protest is untimely or the protest fails to clearly state legally je grounds of 
protest), the contracting agency must submit an agency report to GAO responding to the 
protest and a copy of that report must be provided to the protester. The protester then 
may file written comments on the report. 

During the course of a GAO protest, the Comptroller General may schedule 
informal meetings or conferences to resolve procedural matters and obtain information 
pertaining to the disposition of the protest. GAO may also find that a hearing must be 
held to resolve factual and legal issues raised in the protest. If it decides to hold a 
hearing, GAO will often conduct a pre-hearing conference to (1) decide the issues that 
will be considered at the hearing, (2) identify the witnesses who will testify at the 
hearing, and (3) settle procedural questions. After the hearing, all parties may submit 
written comments. 

After the record is complete, GAO will consider the facts and legal issues raised 
and issue a decision. A copy will be sent to all parties participating in the protest. GAO 
may sustain the protest (that is, find that the agency violated a procurement statute or 
regulation and prejudiced the protester), in which case GAO will recommend appropriate 


corrective action. Conversely, GAO may deny the protest or dismiss the protest without 
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reviewing the matter. GAO issues its decisions within 100 days from the date of protest 
filing. [Ref. 8] 
C: METHODOLOGY FOR NARROWING THE BEST VALUE PROTESTS 
Several assumptions regarding the research must be understood. There were a 
total of 146 decisions rendered by GAO on best value contract awards, and 23 of the 
protests were sustained. Protests are initiated by contractors for many reasons such as 
perceived improprieties in the solicitation, competitive range determination, or evaluation 
of the various proposals. It is important to note that although each of the 146 decisions 
were about a best value contract, some of the protests and sustainments had nothing to do 
with the best value criteria in the solicitation. This study focuses first on protests that 
challenge the very process of determining the best value offeror. The process is called the 
tradeoff process. As previously discussed, FAR Part 15 defines the tradeoff process as 
part of the best value continuum where dollars are traded for incremental value and cost is 
only one factor-not the controlling factor. The tradeoff process is appropriate when it 
may be in the best interest of the Government to consider award to other than the lowest 
priced offeror or other than the highest technically rated offeror. In addition to the 
tradeoff elements, this study focuses on protests that challenged Federal agency pre- and 
post-award changes. These protests alleged that agency actions (changes) invalidated the 
best value source selection. For the purpose of this study, a protest of a best value award 


occurs when (1) a Federal agency evaluation of a tradeoff element 1s challenged (e.g. 


ls 


cost/price, technical merit, past performance, labor rates), or (2) an improper agency pre- 
or post-award change occurs that compromises an otherwise proper best value award. 
D. OVERALL STATISTICS 

The 146 protests and 23 sustainments of best value awards were narrowed to the 
108 protests and 21 sustainments that met the researcher’s definition of a best value 
award protest (improper agency evaluation of a tradeoff element or improper pre- or post- 
award changes). The other protests were simply challenges of agency evaluations of 
peripheral elements that were in a best value solicitation only by chance (e.g. competitive 
range determinations, constitutionality). Of these 108 protests, 87 were denied or 
dismissed, and 2] were sustained yielding a sustainment (success) rate of 19.44 percent. 
This compares to an overall sustainment rate of 12 percent for all protests in 1997. [Ref. 
9] The sustained protests will be categorized and discussed for the remainder of the 
chapter. The Appendix lists all 146 protests reviewed during the research. 
E. SUSTAINMENT CATEGORIES 

The 21 sustainments address specific allegations regarding Federal agency 
evaluation of best value proposals. Nearly all of the protests cited more than one reason 
for protest. The categorical breakdown outlined below represents the category and 


success rate of the reasons for sustainment of the 21 best value protests. 
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1. Improper Agency Evaluation of Past Performance 

Past performance information is one indicator of an offeror’s ability to perform a 
contract successfully. Relevant past performance can be used effectively by contracting 
officers to predict the capability of offerors. Improper Federal agency evaluation of past 
performance was a sustaining argument in six (5.56 percent) of the 108 protests. These 
six sustainments represent 28 percent of the 21 successful best value protests. 

2: Improper Agency Evaluation of Technical Merit 

Evaluation of technical merit is key to best value tradeoff and source selection. 
Technical merit is a prime indicator of quality. Current regulations mandate a technical 
assessment whenever tradeoffs are performed. [Ref. 1: 15.305] Improper Federal agency 
evaluation of technical merit was a sustaining argument in seven (6.48 percent) of the 108 
protests. These seven sustainments represent 33 percent of the 21 successful best value 
protests. 

3. Improper Agency Evaluation of Cost/Price 

Cost/Price is clearly one of the most important elements in negotiated 
procurements. It is also a central tradeoff element in best value source selection. 
Improper Federal agency evaluation of cost/price was a sustaining argument in six (5.56 
percent) of the 108 protests. These six sustainments represent 28 percent of the 21 


sustainments. 
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4. Improper Agency Evaluation of Labor Qualifications 

Labor qualifications are often an important tradeoff element in best value source 
selection. Special or unique qualifications bring value to an offeror’s proposal as well as 
cost. Improper Federal agency evaluation of labor qualifications was a sustaining 
argument in two (1.56 percent) of the 108 protests. These two sustainments represent 
9.52 percent of the 21 successful protests. 

5: Improper Agency Pre- or Post-Award Changes 

Improper agency pre- or post-award changes are contracting officer actions which 
undermine and invalidate otherwise proper best value awards. Improper Federal agency 
pre or post-award change was a sustaining argument in three (2.78 percent) of the 108 
protests. These six sustainments represent 14 percent of the 21 sustainments. 


Figures 3-1 and 3-2 outline the overall and sustainment statistics. 
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F. CONCLUSION 

This chapter displays the dominant reasons for success in the 21 best value 
sustainments in 1997. Improper evaluation of tradeoff elements (past performance, 
technical merit, cost/price, and labor qualifications) made up 18 (85.71 percent) of the 21 
sustained protests. Improper pre- or post-award changes were sustaining arguments in 
only 3 of the 21 sustainments; however, it 1s important to note that this offense was 
alleged only three times in the pool of 108 best value protests. In other words, even 
though improper changes were alleged infrequently, they were successful every time. 
Chapter IV discusses and analyzes the six protests that were sustained for agency 


improprieties in past performance evaluation. 
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IV. PAST PERFORMANCE 

A. INTRODUCTION 

Past performance information is one indicator of an offeror’s ability to perform a 
contract successfully. [Ref. 1: 15.305] Improper evaluation of past performance was a 
sustaining argument in six (28 percent) of the 21 successful best value protests in 1997. 
This chapter details all six protests and illustrates common elements leading to 
sustainment. The protests in which improper evaluation of past performance was a 
sustaining element are American Combustion Industries, Inc.; International Business 
Systems, Inc.; McHugh/Calumet, A Joint Venture; Mechanical Contractors, S.A.; 
NavCom Defense Electonics, Inc.; and ST Aerospace Engines Pte. Ltd. The six protests 
are divided into two categories: 

1. Improper evaluation of awardee or protester past performance; 

2. Failure to conduct RER discussions regarding adverse past performance 

reports. 

One of the protests included both of the above categories as successful sustaining 
arguments. This chapter discusses current and past regulations regarding past 


performance, describes the six sustained protests, and provides analyses of Federal 


agency miscues when evaluating past performance. 
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B. PAST PERFORMANCE AND FAR PART 15 

Past performance is, by statute, a mandatory evaluation element of all negotiated 
source selections. It is also a key tradeoff element in best value contracting. Before 
analyzing best value sustainments for improper evaluation of past performance, it is 
important to understand the effect of the FAR Part 15 changes that were published on 30 
September 1997. Past performance evaluation, both before and after the rewrite, has 
several basic precepts. First, all factors and significant sub-factors regarding past 
performance must be stated in the solicitation. In addition, the offerors shall be provided 
with an opportunity to identify past or current similar contracts as well as an opportunity 
to comment on adverse reports. [Ref. 1: 15.304] While the rewrite was not in effect 
when GAO issued most of the 1997 decisions, the planned changes were well known. 
With respect to past performance, the FAR Part 15 changes were few. One significant 
difference is that before the rewrite, firms lacking relevant past performance history 
received a Neutral evaluation for past performance. [Ref. 13: 15.608] The rewrite 
modified this part of the regulation to state that an offeror without a record of relevant 
past performance or for whom information on past performance is not available, may not 
be evaluated favorably or unfavorably on past performance. In addition, the rewrite 
states that the Government will not rely on adverse past performance information about 
which the contractor's have not had an opportunity to comment. [Ref. 1: 15.305] Even if 


the award is to be made without discussions, offerors must be given the opportunity to 
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clarify certain aspects of their proposals (e.g. relevance of an offeror’s past performance 
information and adverse past performance information) to which the offeror has not 
previously had an opportunity to respond. [Ref. 1: 15.306] The newest exchange 
(communications) allows exchanges prior to the establishment of the competitive range to 
assist in the competitive range determination, address adverse past performance reports, 
and clear ambiguities. The implication for the contracting officer is that there is now an 
avenue outside the clarification/discussion realm that allows tailored exchanges with 
offeror(s) about adverse past performance references. 

The remainder of this chapter analyzes the six protests that were sustained due to 
improper agency evaluation of past performance in 1997. 
C. IMPROPER EVALUATION OF AWARDEE/PROTESTER PAST 

PERFORMANCE 

1. International Business Systems, Inc. 

International Business Systems, Inc., B-275554, Comptroller General of the 
United States, 3 March 1997, (IBSI) was sustained due to agency improprieties in 
evaluating past performance. IBSI protested the Veteran's Affairs (VA) award of a fixed- 
priced contract for a replacement telephone system for the VA Medical Center, Wilkes- 
Barre, Pennsylvania. The contract competition was limited to participants in the Small 


Business Administration's (SBA) Section 8(a) small, disadvantaged business program. 
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The VA found Dulles Networking Associates, Inc. (DNA) to be the offeror whose 
proposal was determined to be most advantageous to the Government. 

The evaluation scheme involved a two-step review. First, technical proposals 
were to be evaluated on a pass/fail basis. Second, the technically — offers would 
be reviewed for price and past performance which were weighted equally in the Request 
for Proposals (RFP). Adjectival ratings were to be assigned to each offeror for past 
performance. Three proposals including IBSI and DNA were found technically 
acceptable. DNA offered the lowest price ($ 2.4 million) and was issued a past 
performance rating of Excellent. IBSI offered a price of ($ 2.9 million) and was issued a 
past performance rating of Good. 

IBSI argued in its protest that DNA's excellent past performance rating was 
unreasonable because DNA had no direct experience furnishing and installing telephone 
systems. The contracting officer (CO) reevaluated and issued a Neutral rating to DNA in 
accordance with the RFP. In reevaluating IBSI's proposal, the CO identified two 
references directly applicable to the solicitation. She based her review on only one 
reference because the second reference, involving the installation of a similar telephone 
system at a VA Medical Center in Massachusetts, was not included because the 
individual within the agency responsible for completing the reference form failed to do 


so. With only the one reference, IBSI was again given a Good past performance rating. 
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The CO then determined that a Neutral rating (DNA) was essentially equal to one Good 
rating (IBSI) and DNA's lower price held sway. 

IBSI argued that the agency was required to consider the second reference (its 
installation of the Massachusetts Medical Center telephone system). VA countered and 
GAO supported the assertion that there was no legal requirement that all past 
performance references be included in a valid review of past performance. The winning 
factor for IBSI was in the fact that the very same agency, contracting officer, and same 
services were involved in the contract award and administration of the telephone 
installation at the Massachusetts VA Medical Center (missing second reference). In fact, 
the contracting officer stated that IBSI's performance had been "exemplary" in a letter 
written four months prior to the award decision regarding the Pennsylvania VA Medical 
Center. Under these circumstances, GAO held that the agency knew of the past 
performance and unreasonably failed to consider IBSI's positive performance in the 
second reference. 

Even though GAO found the VA evaluation of IBSI’s past performance to be 
unreasonable, competitive prejudice was an essential element that had to be established 
before the improper evaluation would be sustainable. GAO concluded that IBSI was 
likely prejudiced by the agency’s failure to include the past performance in the second 
reference (VA Medical Center in Massachusetts). The “exemplary” performance by IBSI 


would have likely garnered at least a rating of Good. The agency would then have to 
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weigh IBSI’s two Good ratings against DNA’s one Neutral rating. This change in past 
performance references could have resulted in a different selection. GAO recommended 
a reevaluation of the proposals including IBSI’s two references. [Ref. 10] 

2. Mechanical Contractors, S. A. 

Like /BSI, Mechanical Contractors, S. A, B-277916, Comptroller General of the 
United States, 27 October 1997, (MECSA) was sustained because of an error by the 
Federal agency. MECSA protested the Panama Canal Commission (PCC) award ofa 
firm fixed-price contract for cleaning and painting of four miter gate leaves in the Panama 
Canal. The RFP called for abrasive blast cleaning and exterior painting above and below 
the water line using hot-applied coal tar enamel. The PCC found Formal Management 
Systems, Inc. (FMS) to be the offeror whose proposal offered the best value to the 
Government. 

The RFP required all offerors to submit a technical proposal and price schedule. 
The technical proposals were to be evaluated under two equally weighted factors: 
technical approach and performance capability. Performance capability was divided into 
five sub-factors including specialized experience and past performance. The total 
technical evaluation scores were to be weighted equally with the offerors’ firm fixed- 
price in an effort to determine the best value to the Government. Three proposals were 


received including those of MECSA and FMS. 
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MECSA proposed the lowest price of $ 2.5 million while FMS submitted a price 
of $ 2.6 million. The key difference between the two offerors existed in the performance 
capability factor. FMS earned 81.55 points in performance capability. MECSA earned 
66.2. Past performance was the significant difference in the two proposals because PCC 
awarded the contract to FMS even though MECSA proposed a lower price. The 
evaluation record showed that FMS had successfully cleaned and coated all PCC miter 
gates for the previous three years. MECSA had weaknesses in experience and past 
performances as the agency stated they had very little coal tar enamel work over the same 
three years. 

MECSA argued that PCC’s evaluation of its and FMS’s past performance was 
unreasonable. Offerors were to list similar projects over the past three years. The 
Evaluation Board reviewed 17 contracts and concluded that three (18 percent) showed 
either late completion or liquidated damages. The board stated that MECSA had been 
late on “various projects” and awarded MECSA only a 50 percent score in past 
performance. GAO found the reasonableness of the score in doubt. According to PCC’s 
post-protest submissions, only five contracts should have been considered similar with 
one (20 percent) being late. It is important to note that although the performance was 
cited as late, MECSA received an overall performance rating of Satisfactory in this 
similar contract. PCC maintained that under the streamlined number of contracts, 


MECSA’s rate of late performance rose from 18 to 20 percent and their decision would 
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have remained unchanged. However, GAO held that with only one suspect similar 
contract, the key downgrading statement regarding late performance on “various projects” 
was now unsupported and unreasonable. 

In contrast, PCC’s evaluation of FMS’s past performance appeared inconsistent 
with the evaluation of MECSA. MECSA pointed out that FMS had serious, documented 
safety violations (including a fatality) on previous projects that had no affect on its past 
performance score. PCC stated that the proposal did not disclose the information, and the 
Evaluation Board did not weigh these facts. However, GAO queried the board regarding 
the fatal safety violation to which the members stated that they did have knowledge about 
the incident at the time of the evaluation. During the protest, PCC reevaluated FMS’s 
proposal, taking the safety violations into consideration, and still maintained that FMS 
represented best value for the Government. 

MECSA also argued that FMS had not been as successful when cleaning and 
painting miter gates as they had originally maintained in their proposal. MECSA pointed 
out that in still another contract, FMS received an unsatisfactory performance rating ina 
safety area. PCC maintained that since FMS had garnered a satisfactory performance 
rating for the entire contract, 1t was reasonable that the safety problem should have no 
affect on the past performance rating. GAO disagreed stating that this approach seemed 
inconsistent with the downgrading of MECSA for late performance under their one late 


contract in which the overall performance rating had been satisfactory. 
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As to a determination of prejudice, GAO stated that PCC’s reevaluation of the 
specialized experience of FMS—taking into account the fatal safety violation—resulted 
in only a small difference in overall adjectival point scores (1.2 points). In light of the 
protester’s lower proposed price, GAO determined that MECSA would have had a 
substantial chance of receiving the award. GAO held that the presence of the multiple 
material errors in the evaluation of past performance and technical merit (to be discussed 
in a later chapter) did prejudice MECSA. Consequently, GAO recommended 
reevaluation. [Ref. 11] 

3. - NavCom Defense Electronics, Inc. 

NavCom Defense Electronics, Inc., B-276163, Comptroller General of the United 
States, 19 May 1997, (NavCom) was similar to JBS] and MECSA because agency miscues 
regarding past performance resulted in sustainment of the protest. NavCom protested the 
U. S. Air Force’s award of a firm fixed-price contract for repair of over 104 line items in 
their AN/ARN-118 Tactical Air Navigation (TACAN) systems. The contract 
competition was a total small business set aside for one base year and four one-year 
options. The Air Force found Integrity Air Services, Inc. (IAS) to be the offeror whose 
proposal was determined to be most advantageous to the Government. 

The TACAN system is a complex collection of five major components: (1) a 
radio receiver/transmitter; (2) a radio receiver transmitter control; (3) a digital to analog 


adapter; (4) an antenna; and (5) a mount. The system contains over 15,000 sub- 
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components. The RFP stated two evaluation factors—performance and price—were of 
equal importance and best value would be determined through a performance/price 
tradeoff. Eight proposals were received. Under price, IAS ranked first with the lowest 
proposed price and NavCom ranked fourth. The RFP required offerors to forward 
information regarding current and past performance on “the same or similar” efforts. Past 
performance was evaluated as Low, Medium, or High risk. All offerors earned risk scores 
of Low regarding past performance as the Air Force determined that all firms had the 
desired “similar or same” experience with the TACAN system. With equal past 
performance ratings, the lower priced proposal by IAS won the award. 

NavCom protested asserting that IAS did not have “same or similar” past 
performance and should have received a risk rating of other than Low. IAS past efforts 
consisted of only four contracts to repair radio power supplies, and none of the contracts 
were valued at greater than $ 1 million. Power supplies were not a line item for repair in 
the RFP. However, the solicitation did state that the contractor must possess knowledge 
of direct and alternating current circuitry, analog and digital signals, timing networks, 
high frequency networks and electronics, pulse width decoding, synchro technology, 
built-in test circuitry, operation of systems bearing and distance measurement signals, 
radio receiving technology, and pulsed transmission technology. [Ref. 12] 

In contrast, NavCom was the only offeror with satisfactory, direct experience with 


the TACAN system. NavCom’s previous TACAN contract was valued at over $ 8 


32 


million. The Air Force acknowledged that the performance risk assessment team was 
fully aware of NavCom’s past experience; moreover, they were equally aware that the 
remaining offerors had no actual TACAN experience. Still, all offerors received 
performance risk ratings of Low. GAO stated that the record lacked any basis upon 
which the Air Force could reasonably conclude that IAS’s repair of power supplies was 
“similar or same.” 

The Air Force stated that consideration was made to the fact that this was a 100 
percent small business set-aside. They maintained that very few small businesses have 
experience with contracts the size of the TACAN effort. Consequently, it became a 
question of policy whether to award contracts to small business to expand the contracting 
base. GAO stated the goal of contracting base expansion does not allow the agency to 
disregard the terms of the RFP. GAO concluded by stating that only NavCom had the 
“same” experience with the TACAN, and the agency provided no evidence in the record 
establishing that IAS had “similar” experience. GAO could not conclude that the award 
to IAS was reasonable, and they found that NavCom had been prejudiced. GAO 
sustained the protest and recommended reevaluation. [Ref. 12] 

D. FAILURE TO CONDUCT MEANINGFUL DISCUSSIONS REGARDING 
ADVERSE PAST PERFORMANCE REPORTS 
Three of the six protests in which improper evaluation past performance was a 


sustaining argument hinged on agency failure to conduct meaningful discussions 


regarding adverse past performance reports. It 1s important to note that the mere absence 
of required meaningful discussions about adverse past performance reports, when 
required, is by itself a sustainable argument. 

l American Combustion Industries, Inc. 

American Combustion Industries, Inc., B-275057.2, Comptroller General of the 
United States, 5 March 1997, (ACT) was a protest in which an agency failure to hold 
discussions resulted in a protest sustainment. ACI protested the Department of 
Commerce’s National Institute of Standards and Technology’s (NIST) award of a fixed- 
price construction contract for two boilers and boiler housing. The contract was for all 
labor, equipment, and material for the supply and installation of two 82,000 pounds per 
hour boilers and construction of an addition to an existing building to house the boilers. 
NIST determined that Green Contracting Company, Inc. (GCC) offered a proposal 
representing the best value to the Government. 

The RFP established six evaluation factors, three of which involved past 
performance. Past performance factors carried a total assigned weight of 70 percent. 

1. Past Performance on Building Construction (30 percent) 

2. Past Performance on Phased Refurbishing (20 percent) 

3. Past Performance of Personnel (20 percent) 

GAO noted improprieties in the evaluation of the past performance of personnel, and this 


will be discussed in a later chapter involving improper evaluation of labor qualifications. 
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The clear past performance impropriety in this protest occurred in the agency’s attempt to 
evaluate the past performance on building construction. The RFP required offerors to 
provide references on past construction projects. In accordance with the RFP, ACI 
identified two construction projects. One was with the National Aeronautics and Space 
Administration (NASA) and the other with James Madison University (JMU) in Virginia. 

NASA advised the agency that ACI was at least 100 days late in beginning work 
and that they had problems with the firm’s project manager. JMU reported that the 
protester was five months late due to slow delivery of a boiler, in part, because of ACI. 
The JMU reference was otherwise positive. The agency held discussions with ACI 
regarding NASA but not JMU. The agency stated that FAR 15.610(c)(6) anticipated an 
eventual implementation of a Government past performance reporting network. [Ref. 13] 
NIST further stated that the FAR requirement to discuss adverse past performance was 
not in effect until the past performance reporting network was implemented. GAO 
rebuked this notion stating that the FAR plainly mandated an opportunity for offerors to 
reply to adverse past performance information. [Ref. 14] 

The protester also alleged improper evaluation of past performance regarding their 
project manager. The solicitation required the offerors to identify certain key personnel 
who would be involved in the boiler/boiler housing work. ACI identified a project 
manager who was regarded highly by the agency. However, the proposed project 


manager did not appear (as requested by the agency) at discussions conducted by NIST. 
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Since the proposed project manager was unavailable for the discussions, ACI sent its 
president and vice-president in his place. Although the transcript of the discussions 
showed different understandings about what was said regarding the project manager’s 
absence, the agency was concerned that the vice-president in attendance would be 
substituted for the project manager for the initial months of contract performance. The 
evaluation record showed several key facts: 

1. The vice-president who was assumed to be a replacement for the project 

manager was viewed as a poor choice because of negative references. 

2. NIST did not discuss its concerns during the discussions. 

3. ACI did not amend its proposal to show a replacement of the proposed project 

manager with the vice-president. 

4. NIST deducted points from ACI under the past performance of personnel 

category. 

GAO held that the agency’s failure to advise ACI of their project manager 
assumption unreasonably denied the protester a chance to resolve the question about 
personnel availability. Given that a major deduction from ACI’s score was due to the 
perceived substitution with the vice-president, GAO held that the protester had been 
prejudiced. GAO also held that the ACI was prejudiced by the lack of discussions 
regarding JMU. GAO recommended discussions, revised BAFOs, and reevaluation. 


[Ref. 14] 
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2: McHugh/Calumet, A Joint Venture 

McHugh/Calumet, A Joint Venture, B-276472, Comptroller General of the United 
States, 23 June 1997, (McHugh) like ACI, was a protest that was sustained because 
discussions were not held about poor past performance references. McHugh/Calumet 
protested the General Services Administration (GSA) award of a fixed-priced contract for 
construction of a new Federal courthouse in Hammond, Indiana. GSA found Huber, 
Hunt, & Nichols, Inc. (HHN) to be the offeror whose proposal was determined to be most 
advantageous to the Government. 

The RFP called for the construction of a 270,000-square foot, four-story, 
limestone courthouse containing seven courtrooms, Federal office space, a cafeteria, and 
a firing range. The evaluation elements were price and technical factors. Each element 
was weighted equally. The technical factors consisted of (1) quality control—40 
percent; (2) past performance on similar proyects—35 percent; and (3) key personnel 
qualifications—25 percent. Regarding price, McHugh’s proposal of $ 49 million was 
lower than HHN’s by over a million dollars. GSA felt that McHugh’s past performance 
on similar projects was lacking compared to HHN’s. The RFP required one-page 
descriptions and references for projects of similar scope completed in the last five years. 
HHN had satisfactorily worked on three courthouses with “directly comparable” or “very 
similar” architectural (wood paneled courtrooms) finishes. McHugh had constructed no 


courthouses during the five-year period, and the descriptions that they submitted were 


seen to be only “similar” by GSA. McHugh did list two courthouses from before the 
five-year period which GSA found to be “not nearly similar” because they did not have 
the same level of quality finishes. 

HHN’s overall advantage was based not only on greater similarity in projects but 
on better past working relationships. HHN received scores of Above Average for working 
relationships on two submitted references and a score of Outstanding on a third. 

McHugh had two scores of Outstanding and one Below Average. Discussions were not 
held regarding the below average score because it was an internal agency reference, and 
GSA maintained that the FAR 15.610 (c)(6) requirement for discussions applied only to 
third party references. GSA also stated that internal agency references were clear and not 
subject to interpretation, so discussions would be pointless. GAO found no basis why 
FAR 15.610 (c)(6) would not apply in this case by stating that nothing in the FAR limits 
its application to a third party. [Ref. 13] GAO also found that McHugh had been 
prejudiced because discussions would have furnished McHugh the opportunity to contend 
that the poor reference was a personality conflict and not performance related. In light of 
the lower price, close competition, and scores of Outstanding on the other references, 
GAO held that McHugh had a reasonable chance of obtaining award if they were afforded 
the opportunity to refute the poor internal agency reference. GAO recommended 


discussions, revised BAFOs, and reevaluation. [Ref. 15] 
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3. ST Aerospace Engines Pte. Ltd. 

ST Aerospace Engines Pte. Ltd., B-275725, Comptroller General of the United 
States, 19 March 1997, (STA Engines) 1s a protest involving the poor past performance of 
an affiliate as well as a failure by a Federal agency to hold discussions. STA Engines 
protested the Department of Transportation’s U. S. Coast Guard (USCG) award ofa 
contract for the overhaul and repair of helicopter reduction gearboxes and torquemeters 
used in the T-56 engines of C-130 aircraft. USCG found Standard Aero Ltd. (Standard) to 
be the offeror whose proposal was determined to be the best value for the Government. 

The RFP stated that the responsible offeror with the best price/technical merit 
combination would be found to be the best value offeror and obtain the award. Technical 
merit consisted of (descending order of importance) past performance, certification, 
industrial capability/capacity, warranty, and engineering and support capability. The 
evaluators stated that STA Engines was downgraded primarily due to concerns about the 
company’s past record of late delivery. USCG pointed out that the protester’s past 
performance was for the most part good except for significantly late deliveries of 
overhauled ship propellers under a separate USCG contract. 

STA Engines maintained that the negative past performance rating was 
unreasonable because they had not participated in the contract in question. STA Engines 
asserted that it was their affiliate ST Aerospace Systems (STA Systems) that had 


performed poorly. STA Engines further maintained that had they the opportunity to 


discuss the poor report, they could have vindicated themselves and won the award. 
USCG countered stating that STA Engines proposal had held the two as affiliated and 
even forwarded the propeller contract as a reference. GAO stated that affiliation alone 
was not a deciding factor regarding STA Engines. GAO pointed out ~~ the relationship 
between the two was key. The critical question was whether the workforce, 
management, facilities, or other resources of one company may affect the contract 
performance of the affiliate. 

GAO held that the agency had provided no evidence that the past performance of 
STA Systems was of any relevance to the potential contract performance of STA 
Engines. Furthermore, the agency was required to raise the issue of the affiliate’s 
performance and relationship with STA Engines during discussions. Specific weaknesses 
should have been pointed out. Had the discussions been held, STA Engines would have 
been able to challenge any negative past performance of any of the other units in the 
parent company--ST Aerospace Group. GAO held STA Engines was prejudiced by both 
the improper attribution of an affiliate’s negative past performance and the failure of 
USCG to hold meaningful discussions. GAO recommended discussions, revised BAFOSs, 
and reevaluation. [Ref. 16] 
Ex ANALYSIS AND CONCLUSION 

The purpose of this analysis is to give the contracting officer an indication of the 


common pitfalls when evaluating past performance in best value contracts. 
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¡e Follow the Solicitation in a Consistent Manner 

It is clear in several protests that failure to follow the solicitation’s evaluation 
scheme led to the improper evaluation of past performance and sustainment. NavCom is 
a good example. In this protest, we see that an RFP placing emphasis on “same or 
similar” past performance must be followed. The RFP was clear in this case where past 
TACAN experience was required, yet the contracting officer seemed to arbitrarily judge 
every offerors’ past performance as equal (low risk). GAO held that the agency 
evaluation was not consistent with the solicitation. This decision stemmed from the 
fundamental requirement for agency evaluations to be reasonable and consistent with 
stated evaluation criteria. GAO cited a previous decision in support of their finding 
(Ogden Support Services, Inc., B-270012.2, 19 March 1997). 

NavCom also shows us that circumventing the RFP for unstated policy goals (e.g. 
increasing the industrial base) 1s not supported by the Comptroller General. In this 
protest such a goal existed, but the RFP contradicted the goal by placing a high premium 
on “same or similar” past performance. The Federal agency could not both follow the 
RFP and achieve their goal of increased industrial capacity. GAO held that the agency 
evaluation was unreasonable. Their holding stemmed from the Comptroller General’s 
assertion that the agency cannot disregard the terms of the RFP in evaluating an offeror’s 
proposal—even though the agency has an outlying goal of industrial base expansion. The 


implication from NavCom is twofold. First, contracting officer judgment is often 
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supported by GAO, but the contracting officer must be reasonable, consistent, and in 
accordance with the RFP. Second, peripheral agency goals do not absolve the contracting 
officer from the requirement to adhere to the solicitation. If an increased industrial base is 
the priority, then the RFP should allow for inexperience and place less the emphasis on 
past performance. However, there is a risk of poor or non-performance with this strategy. 
CICA certified Congress’ intent to maximize competition, but the contracting officer is 
bound to evaluate the proposals reasonably and consistently in accordance with the RFP. 

2. Make Use of Relevant, Known Facts About Offerors 

MECSA demonstrates problems with agency use ofrelevant data. The awardee in 
this protest had several previous safety violations that were known to the evaluation 
board. Still, the agency chose to ignore the violations and did not downgrade the offeror. 
The key here is relevance. GAO found that in the dangerous activity of blasting, 
preserving, and painting Panama Canal miter gates above and below the water line, safety 
was a relevant issue. The evaluation board ignored these facts in its past performance 
evaluation, and it was a sustaining argument for the protester. GAO held that the Federal 
agency was not consistent in its evaluation. The holding was rooted in a previous decision 
(Boeing Sikorsky Aircraft Support; B-277263.2, B-277263.3; 29 September 1997) that 
stated offerors must receive the fair and considered judgment of the agency. The 
implication for the contracting officer is that inconsistency in application of evaluation 


criteria is an obvious but fatal flaw in past performance evaluation. 
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IBSI reinforces this theme. This protest exposes the disregarding of known, 
positive information by an agency that could have helped the protester. Not only had the 
Veteran’s Administration (VA) improperly given the awardee a good past performance 
rating in the light of no direct experience, but an agency employee forgot to fill out a 
reference form and reduced the protester’s pool of positive references from two to one. It 
is important to note that in JBS/, the mistake by the agency was not the cornerstone to 
sustainment. There is no legal requirement that all past performance references be 
included in a valid evaluation. Rather, the fact that the missing second reference involved 
the same services, Federal agency, and contracting officer caused the Comptroller 
General to find that the previous “exemplary” performance of the protester was a known 
fact to the VA. GAO held that it was unreasonable for the Federal agency to ignore this 
known and relevant past performance. The Comptroller General supported its decision 
by citing a previous protest sustainment (Marine Diesel, Phillyship; B-232619, B- 
232619.2; 27 January 1989) where the Navy chose not to consider unsatisfactory past 
performance of an awardee involving similar services and the same command because the 
awardee did not include the controversial contract on its list of references. The 
implication for the contracting officer is that an agency cannot simply choose to ignore 
relevant information. The failure to utilize relevant data could result sustainment. 

Relevance is a double-edged sword. In STA Engines, the USCG applied irrelevant 


past performance data about an affiliate and unreasonably penalized the protester. This is 


43 


not to say that the performance of affiliates 1s always irrelevant. GAO stated that the 
relationship determines the relevance. If the resources and actions of one affiliate can 
affect the potential contact performance of another, then the affiliate’s past 
performance—positive or negative—is relevant. GAO held that the agency had not 
demonstrated that the past performance of the protester’s affiliate was relevant. The 
holding stems from a previous protest (Contract Services Co., Inc., B-246604.2, 11 June 
1992) where GAO stated that it would be inappropriate to consider the affiliate’s record 
where that record did not bear on the likelihood of successful performance by the 
protester. The implication for the contracting officer is that an affiliate’s past 
performance must be carefully screened for relevance. Relevant past performance data 
should be used, and irrelevant data should be disregarded. 

3. Hold Discussions Regarding Negative Past Performance 

Discussions regarding poor past performance references must be held to give the 
offerors an opportunity to explain the situation. This was a frequent, sustaining error by 
Federal agencies in three protests. AC] shows that although the FAR is and was clear in 
this matter, mistakes can be made. The agency (NIST) held discussions with the 
protester regarding a poor reference from NASA but elected not to do so regarding James 
Madison University (JMU). The reason was the agency’s perceived delay in the FAR’s 
requirement to discuss poor references until an anticipated past performance reporting 


network was implemented. GAO held that discussions were required. This finding was 
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rooted in the fact that the FAR (15.610) was clear, and the absence of the anticipated 
reporting network in no way relieved a Federal agency from its duty to inform an offeror 
of a poor past performance report. The implication for the contracting officer is that poor 
past performance reports must be addressed to the offeror to provide an opportunity for 
comment. 

McHugh demonstrates that an assumption about the source of a reference can be 
troublesome. GSA believed the requirement to discuss a negative past performance 
existed only for third party references. Internal agency references were assumed to be 
clear and not subject to interpretation; therefore, discussions were pointless and not 
required. However, the Comptroller General found no basis why the FAR requirement 
for discussions would not apply in this case and sustained the protest. This holding stated 
that nothing in the FAR limited the application of the requirement for discussions to third 
party references. Contracting officers should be wary of past performance information 
that they consider indisputable (e.g. intra-agency references). The source of a poor report 
does not affect the contracting officer’s requirement to provide the offeror with a chance 
to address the situation. 

As previously stated, STA Engines demonstrates that an affiliate’s past 
performance may not necessarily be relevant to a sister company’s ability to perform the 
terms of a contract. STA Engines also shows us that the holding of discussions may have 


precluded the improper use of the affiliate’s past performance. STA Engine’s had not 


been given the opportunity to respond to the agency’s contention that their affiliate’s 
performance was relevant to their ability to perform. If discussions had been held, the 
problem may have been averted. In any case, GAO held that the lack of discussions was 
a sustaining argument for the protester. GAO bolstered its holding by citing a previous 
protest (Alliant Techsystems, Inc., Olin Corporation; B-260215.4, B-260215-5; 4 August 
1995) that stated for discussions to be meaningful, an agency must point out significant 
weaknesses in a proposal that would prevent the offeror from having a reasonable chance 
for award. The contracting officer must realize that discussions assist the agency by 
identifying potential improprieties in the evaluation before award is made. Had 
discussions been held, objections by the protester could have caused the agency to review 
the situation further and make proper evaluation decisions. 

A final observation can be drawn regarding required discussions. In two of the 
three cases, the Federal agencies maintained that the protester knew about the poor past 
performance and that their knowledge released the agencies from their obligation to hold 
discussions. In both cases, GAO held that knowledge of negative past performance was 
not the issue. GAO indicated that contracting officers must inform an offeror of the 
impact of the reference. Federal agencies must state if an offeror’s past performance 
detracts from their ability to win the award. Discussions (communications) must be held. 

Contracting officers should take many valuable lessons from this chapter. 


Consistent, relevant, evaluation of a proposal in accordance with the solicitation is 
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crucial. Underlying policy goals do not allow a Federal agency to disregard the terms of 
a RFP. All known facts within the confines of the evaluation criteria must be used when 
considering past performance. Irrelevant facts must be excluded. Finally, discussions 
must be held to afford each offeror an opportunity to refute a negative past performance 


report even if the offeror is aware of the poor reference. 
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N: TECHNICAL MERIT 

A. INTRODUCTION 

Evaluation of technical merit is a cornerstone of best value tradeoff and source 
selection. Current regulation dictates that when tradeoffs are performed, the source 
selection records shall include (1) an assessment of each offeror’s ability to accomplish 
the technical requirements and (2) a summary, matrix, or quantitative ranking along with 
appropriate supporting narrative of each technical proposal using the evaluation factors. 
[Ref. 1: 15.305(a)(3)] This chapter focuses on Federal agency technical assessment, 
ranking, and supporting narratives when evaluating proposals in an effort to make best 
value awards. The seven protests in which improper evaluation of technical merit was a 
sustaining argument are Cygnus Corporation; HG Properties A, L. P.; J. A. Jones 
Management Services, Inc.; JW Associates, Inc.; International Data Systems, Inc.; 
Mechanical Contractors, S. A., and Technology Services International, Inc. The seven 
protests are divided into two categories: 

1. Improper evaluation of technical merit; 

2. Improper evaluation of technical merit combined with inadequate supporting 


documentation. 
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This chapter discusses the differences in governing regulations as a result of the FAR Part 
15 rewrite, outlines the seven sustained protests, and provides analysis and conclusions 
regarding the reasons for sustainment. 

B. TECHNICAL MERIT AND FAR PART 15 

Only a few of the FAR requirements regarding technical evaluation changed due 
to the Part 15 rewrite. One key difference is the requirement for appropriate supporting 
narrative to accompany the previously required summaries (strengths, weaknesses, and 
basis for award), matrices, and quantitative rankings when making a best value source 
selection. The FAR Part 15 rewrite states that the rationale for tradeoffs must be 
documented in the contract file. While the stated requirement for tradeoff rationale was 
not in force at the time of these protests, it appears that GAO may have intended to send a 
message regarding the importance of these narratives when they firmly stressed the need 
for supporting information in three of the protest decisions. [Ref. 1: 15.101-1] 

In a separate issue, the Part 15 rewrite encouraged open exchanges (clarifications, 
negotiations/discussions, and communications) in an effort to achieve best value. The 
term communications was added to allow better dialog leading the establishment of 
competitive ranges. Clarifications and discussions remained essentially unchanged in 
their definition and application. The distinction between clarifications and discussions is 
a key element in one of the protests in this chapter. It is important to note that 


clarifications are limited exchanges of information to clarify ambiguities when award 
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without discussions is contemplated. Discussions are negotiations after the establishment 
of the competitive range. Discussions are required to be meaningful by indicating 
significant weaknesses, deficiencies, and other aspects that could be altered so that an 
offeror may materially enhance the potential for award. Unlike clarifications, discussions 
may result in an offeror modifying their proposal. [Ref.1: 15.306] 
E. IMPROPER TECHNICAL EVALUATION 

1. Cygnus Corporation 

Cygnus Corporation, B-275181, Comptroller General of the United States, 29 
January 1997, (Cygnus) is a protest where GAO held a Federal agency evaluation of 
technical merit was improper; consequently, the Comptroller General issued a 
sustainment. Cygnus Corporation protested the Department of Health and Human 
Services (HHS) award of a cost-plus-fixed-fee contract for the operation and management 
of the National Adoption Information Clearinghouse. The RFP reserved the award for 
small businesses and outlined one base year with two, one-year options. HHS determined 
the proposal of Caliber Associates, Inc. (Caliber) to represent the best value to the 
Government. 

The RFP stated that the combination of the quality of the technical proposal and 
past performance was of more importance than proposed cost. The solicitation identified 
five criteria for evaluation: (1) quality of technical approach—30 points; (2) staff 


qualifications—20 points; (3) adequacy of manpower, resources, and management plan— 
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20 points; (4) understanding of the statement of work—15 points; and (5) corporate 
experience—15points. HHR received six proposals and found three to be within the 
competitive range. Caliber scored higher than Cygnus, and based on the results of the 
initial and BAFO scores, the agency contract specialist prepared a Negotiations Summary 
Memorandum recommending award of the contract to Caliber. Caliber received the 
award, and Cygnus protested. 

Cygnus argued that HHR improperly downgraded their proposal in the Quality of 
Technical Approach factor because the agency failed to consider added information from 
the protester. The RFP required the offerors to provide an indication of possible 
problems they may encounter in the performance of the contract as well as possible 
resolutions. Cygnus failed to provide this information in their proposal. The agency 
report stated that the omission of this information was the main factor causing deduction 
of points from Cygnus’ score. The agency did not hold discussions because they stated 
that such discussions would result in technical leveling. However, Cygnus did include 
the required information in their BAFO. The record showed that Cygnus’ score was not 
adjusted after the receipt of their BAFO. The evaluator stated that the information on the 
BAFO was evaluated but was not sufficient to change the score. GAO was skeptical. 
The score sheets still contained narrative stating that no discussions were held about the 
missing information regarding potential problems. The record provided neither 


acknowledgment of receipt of the information nor evidence of its use in evaluating the 
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BAFO. For this reason, GAO held that the HHR evaluation of Cygnus’ proposal was 
unreasonable and that they were prejudiced because the omitted information constituted 
the main deduction from their score. GAO sustained the protest and recommended 
reevaluation of Cygnus’ BAFO and comparison to the other offerors’ BAFOs. [Ref. 19] 

2. Matter of HG Properties A, L.P. 

Like Cygnus. HG Properties A [EESB 273 2, B27 922,8 2737, 
Comptroller General of the United States, 29 October 1997, (HG) is a protest that was 
sustained because of improper technical evaluation by a Federal agency. HG Properties 
protested the U.S. Forest Service award of a contract for over 20,000 square feet of office 
space for a Forest Service supervisor’s office located 2.5 miles from Libby, Montana. 
The Solicitation for Offerors (SFO) anticipated a 10 year lease with two, five-year 
options. The Forest Service found that Mountain States Leasing-Libby (MSL) offered a 
proposal representing the best value to the Government. 

The SFO contained specifications outlining the requirements of the building 
(architectural, mechanical, electrical, plumbing, utilities, maintenance, and service 
requirements). One written requirement was that the computer room space must be 
located away from areas housing microwave equipment and radio transmitters. A 
drawing was provided--not to dictate design--but to show desired space relationships and 
predicted traffic patterns. The SFO also stated that if any inconsistency between the 


written requirements and the conceptual drawing existed, the written requirements would 


govern. Finally, the award would be made on a best value basis, and technical evaluation 
factors were equal to price. Price evaluation was to be on the basis of total annual price 
per square foot of occupiable space. 

HG offered a lower price per square foot of $ 12.90. MSL offered $ 12.93. 
However, MSL”s overall point total was better than HG’s (846 (MSL) compared to 757 
(HB)}—1000 point maximum). The higher point total for MSL was primarily due to their 
high score for efficient layout. HG was downgraded because under their layout, supplies 
could not easily be moved. The agency determined that MSL’s offer met all aspects of 
the SFO while HG’s did not meet the basic floor plan. In fact, MSL’s layout was very 
close to the sample layout in the SFO. The agency found that the cost difference between 
MSL and HG was minimal, so MSL was awarded the contract even though they proposed 
a higher price per square foot. HG protested. 

HG argued that the Forrest Service failed to evaluate MSL’s proposal as required 
in the SFO. Specifically, MSL’s layout did not meet the requirement of separating the 
computer room from the telecommunications room. On the contrary, the computer room 
was actually contained within the telecommunications room. The agency stated that 
MSL had simply followed the layout in the SFO and that the minimal needs of the agency 
were represented in the drawing. However, the written direction in the SFO was for 


separation of the computer room from telecommunication and microwave equipment. 
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According to the SFO, the written direction was overriding in the event of a discrepancy 
with the conceptual drawing. 

GAO held that MSL’s proposed layout was not rational and in accordance with 
the stated evaluation criteria (written requirements). The Comptroller General found that 
HG had been prejudiced because they had the lower price per square foot and a layout 
that matched the SFO written direction. GAO recommended that the Forest Service 
amend their solicitation to reflect their actual minimum needs regarding the location of 
the computer room, reopen negotiations with competitive range offerors, and allow them 
to respond. to the amended SFO. [Ref. 20] 

3. International Data Systems, Inc. 

International Data Systems, Inc., B-277385, Comptroller General of the United 
States, 8 October 1997, (IDS) is a protest brought because the agency improperly 
evaluated the technical merit of a delivery schedule and also failed to disclose proposal 
problems during discussions. International Data Systems protested the Department of 
Interior (DOI) award of a fixed-price, indefinite delivery/indefinite quantity contract for 
personal computers. The contract time period was to span six months. DOI found 
Applied Computer Technology (ACT) to be the offeror representing the best value to the 
Government. 

The RFP stated that a Technical Evaluation Panel (TEP) would consider the 


following evaluation criteria: conformance with technical specifications—40 percent, 
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past performance—40 percent, and price—20 percent. Discussions were not to be held. 
Forty-two offers were received with only seven in the competitive range. The TEP 
ranked all seven finalists with the protester earning first-place and ACT second. The 
contracting officer reviewed the seven proposals in greater detail and discovered that the 
protester offered a 30-45 day delivery schedule. The RFP called for a 15-day delivery. 
When establishing the competitive range, the contracting officer had previously rejected 
other proposals that did not meet delivery requirements. She left the protester in the 
competitive range but did not inform them of the delivery problem. Up to this time, the 
contracting officer believed that she had held only clarifications with the offerors in the 
competitive range. She decided not to inform the protester about the schedule problem in 
order to avoid holding discussions and delaying the procurement. The DOI found ACT’s 
proposal to be the best value and awarded them the contract. International Data Systems 
protested. 

IDS argued that the agency’s unwillingness to inform them of the problem 
constituted a failure to hold meaningful discussions. IDS maintained that if the problem 
had been brought to their attention, it could have been immediately remedied because 
they stated it was simply a typographical error. DOI countered that they had engaged in 
clarifications with the seven offerors and not discussions. Consequently, IDS was not 
entitled to discussions. The governing regulation (FAR Part 15—pre-rewrite) permitted 


contracting agencies to award on initial proposals without discussions provided that the 
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solicitation made the situation clear. Clarifications were defined as communications with 
an offeror for the sole purpose of eliminating irregularities or apparent clerical mistakes 
in a proposal. Discussions involved essential information for determining the 
acceptability of a proposal or provided the offeror an opportunity to revise the proposal. 
Unlike discussions, clarifications do not give an offeror the opportunity to make proposal 
revisions. 

GAO held that the clarifications held by the agency with some of the offerors 
were actually discussions for two reasons. First, the communications concerned 
information essential for determining the acceptability of the offerors’ proposals. 

Second, the offerors were given an opportunity to submit BAFOs thereby modifying their 
proposals. The fact that discussions were held meant that DOI was obligated to raise the 
schedule issue with IDS. When discussions are held, they are required to be meaningful. 
Discussions are not meaningful unless they address aspects of an offeror’s proposal that 
must be corrected for the offeror to have a reasonable chance of obtaining award. GAO 
held that meaningful discussions were required but not held with IDS because the 
erroneous delivery schedule had not been addressed. GAO stated that the lack of 
required, meaningful discussions coupled with an improper pre-award change by the 
agency (discussed in a later chapter) prejudiced IDS. The protest was sustained. Since 
the computers were already delivered and accepted, GAO recommended reimbursement 


of proposal and protest costs to IDS. [Ref. 21] 
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4. Technology Services International, Inc. 

Technology Services International, Inc., B-276506, Comptroller General of the 
United States, 21 May 1997, (TSI) 1s a protest in which a proposed scheduling system is 
successfully challenged. Technology Services protested the U.S. Air Force award of a 
fixed-price contract for grounds maintenance services at Hickam Air Force Base, Hawaii. 
The contract was for a six-month base period with five option years. The Air Force 
determined Oahu Tree Experts (OTE) to be the offeror representing the greatest value to 
the Government. 

The RFP defined this as a performance based contract with a best value evaluation 
scheme consisting of the following criteria: 

1. Technical (Grounds Maintenance) 
e Quality Control 
e Equipment, Material, and Supplies & Work Procedures 
e Customer Service 

2. Management (Manpower Planning) 


Past Performance 


U) 


4. Price 


The first three criteria were equally weighted, and in combination, were outlined to be 


more important than price. 
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OTE received an Unacceptable rating in Technical (Grounds Maintenance)}— 
Quality Control because the offeror did not include a work scheduling system as required 
by the RFP. OTE informed the agency that such a schedule would be developed. Asa 
result of this promise, OTE received an Exceptional rating (highest possible) in Quality 
Control. OTE and TSI received similar ratings in the various evaluation criteria. OTE’s 
proposed price of $4.8 million was nearly $2 million less than the protester’s $6.5 
million. Both proposals were found to be essentially equal from a technical perspective, 
so OTE’s lower price won the contract award. Technology Services protested. 

The protester argued that the agency technical evaluation of OTE’s proposal was 
unreasonable. The protester maintained that OTE’s BAFO did not provide a work 
scheduling system that met the requirements in the RFP. Their system failed to show the 
day and time that the work was to be performed. The Air Force countered that OTE had 
promised to develop such a schedule and that their promise was satisfactory. GAO 
disagreed. The solicitation stated that an Exceptional rating was supposed to indicate 
that a proposal had exceeded the standards outlined in the RFP. GAO held that the 
assignment of such a high rating was unreasonable. Moreover, the Comptroller General 
stated that they could not conclude that the two proposals were technically equal. GAO 
sustained the protest and recommended that OTE’s contract be terminated and awarded to 


TSI. [Ref. 22] 
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D. IMPROPER EVALUATION OF TECHNICAL MERIT COMBINED 

WITH INADEQUATE SUPPORTING DOCUMENTATION 

1. Mechanical Contractors, S. A. 

Mechanical Contractors, S. A., B-277916, Comptroller a ofthe United 
States, 27 October 1997, (MECSA) is discussed in Chapter IV because one sustaining. 
argument involved improper evaluation of past performance. MECSA was also sustained 
because of an error by the Federal agency in evaluating technical merit. Mechanical 
Contractors protested the Panama Canal Commission (PCC) award of a firm fixed-price 
contract for cleaning and painting of four miter gate leaves in the Panama Canal. The 
RFP required all offerors to submit a technical proposal and price schedule. The 
technical proposals were to be evaluated under two equally weighted factors: technical 
approach and performance capability. The total technical evaluation scores were to be 
weighted equally with the offerors’ firm Ax pos in an effort to determine the best 
value to the Government. MECSA proposed the lowest price of $2.5 million while FMS 
submitted a price of $2.6 million. Under technical approach, FMS earned a score of 72.7 
points compared to 60.85 for MECSA. Despite MECSA’s lower price, FMS was 
awarded the contract. 

The protester challenged the award arguing that the agency technical evaluation of 
its proposal was not adequately supported. There was very little contemporaneous 


supporting documentation regarding MECSA’s technical evaluation. Under technical 
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approach, the evaluation board stated only that MECSA had “very little coal tar enamel 
(CTE) experience over the past three years.” [Ref. 11] 

Regarding the experience statement, the protester maintained that PCC failed to 
evaluate their proposal properly. The RFP stated that substantially equal importance 
would be given to an offeror’s CTE experience or an offeror’s possession of a special 
certification (SSPCat QP-2). The protester’s proposal outlined the use of a subcontractor 
holding this certification. The protester contended that since they (through their 
subcontractor) held the QP-2 certification, which was supposed to be substantially equal 
to CTE experience, their score for technical approach should have been essentially equal 
to FMS’s. 

PCC countered that the solicitation did not guarantee equal or equivalent 
treatment of CTE experience and QP-2 qualification. PCC also asserted that the 
evaluation board did give favorable consideration to the subcontractor certification. GAO 
agreed that the evaluation board would have been reasonable for a slight downgrading of 
MECSA regarding their limited direct CTE experience. However, GAO held that the 
lack of contemporaneous supporting documentation cast doubt as to whether the 
evaluation board actually gave MECSA credit for the subcontractor certification. The 
only documentation that accompanied the numerical score stated “very little CTE 
experience.” GAO held that there was no evidence of consideration of the subcontractor 


certification. GAO held that the presence of the multiple material errors in the evaluation 
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of technical approach and past performance (Chapter IV) did prejudice MECSA. 
Consequently, GAO sustained the protest and recommended reevaluation. [Ref. 11] 
2. J. A. Jones Management Services, Inc. 

Like MECSA, J. A. Jones Services, Inc., B-276874, Conable General of the 
United States, 24 July 1997, (Jones) is a protest where the combination of improper 
technical evaluation and the lack of proper agency documentation resulted in sustainment. 
Jones Management Services protested the U.S. Army Corps of Engineers’ award of a 
fixed-price contract for base repair and construction at the Bluegrass Army Depot in 
Lexington, Kentucky. The RFP called for offerors to submit their best price/cost and 
technical terms in the initial proposals because the agency intended to issue award 
without discussions. The contract was to be for an 18-month period with two option 
years. The U.S. Army Corps of Engineers (USACOE) found Intersteel, Inc. (Intersteel) 
to be the offeror whose proposal represented the best value Government. 

The solicitation listed the following evaluation criteria (descending order of 
importance): (1) management ability; (2) subcontracting support capability; (3) related 
experience; (4) unit price coefficient; (5) technical staff capability; and (6) financial 
ability. Regarding price, each offeror was required to include a coefficient or percentage 
factor to be applied to specified fixed rates for various prices contained in a unit price 
book (enclosed with the RFP). Jones Management Services challenged the award stating 


that the majority of the technical point ratings were inaccurate, unsupported, or otherwise 
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improperly based on undisclosed evaluation criteria. Jones Management Services 
maintained that the agency evaluation and supplemental report provided explanations 
with information that was either cursory, missing, or “squarely at odds” with the proposal 
information. The record revealed that the only supporting documentation consisted of 
bullet statements annotated on the score sheets. In addition, the scant cursory narratives 
did not match the scoring deductions. The protester charged that strengths and 
weaknesses assigned to the offerors bore no relationship to the evaluation criteria outlined 
in the RFP. To further support its position, the protester pointed that the evaluation 
record did not reference a single paragraph of either the protester or the awardee 
proposals. 

Regarding the allegation of inaccurate evaluation, the agency countered that the 
bullet narratives on the score sheets were only impressions and did not wholly represent 
the justification for the scores. However, GAO held that exclusion of these bullets left 
the agency with absolutely no supporting documentation. The Comptroller General held 
that the Corps of Engineers’ evaluation was both inaccurate and unsupported by adequate 
documentation, narrative, or information. GAO continued stating: “Here, the dearth of 
evaluation narratives or other point scores, which have been sufficiently shown by the 
protester to be inaccurate in numerous areas, lead us to the conclusion that that the Source 
Selection Authority could not and did not make a reasonable selection decision ...the 


agency has offered no substantive response to the protester’s concerns in response to the 
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protest.” GAO sustained the protest and recommended reevaluation of all proposals. 
[Ref. 17] 

3. JW Associates, Inc. 

JW Associates, Inc., B-275209, Comptroller General of the United States, 30 
January 1997, (JW) is the third protest in this chapter where poor evaluation 
documentation was the basis for a sustaining argument. JW Associates protested the 
Forest Service award of a small business set-aside contract for an environmental impact 
statement for the Cold Springs Analysis Area on the Medicine Bow National Forest in 
Wyoming. The Forest Service determined that Natural Resources Management 
Corporation (NRMC) offered the proposal representing the best value to the Government. 

The RFP listed five evaluation criteria in the order of importance (descending 
order of importance) with the first two categories termed as Very Important and the last 
three as Important: 

Very Important 

1. Qualifications of the Firm 

2. Qualification of the Personnel 

Important 

3. Past Experience of the Firm and the Employees Assigned to the Project 

4. Geographic Location 


5. Price 
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The solicitation also stated that the award would be based on a technical/cost tradeoff 
within the pool of technically acceptable proposals. Each member of the evaluation board 
(made up of three forest service experts) assigned technical scores to the criteria for each 
proposal. The three scores were averaged for each offer, and the proposals were ranked. 
JW was the highest technically rated offeror but ranked sixth in price. NRMC ranked 
first in price. The board prepared a memorandum for the record containing a one- 
paragraph summary of each proposal. The contracting officer concluded that NRMC’s 
proposal represented the best value to the Government and made the award. JW received 
a debriefing letter along with a copy of the awardee’s technical proposal. JW protested. 

JW argued that the Forest Service’s evaluation of the offerors’ technical proposals 
was improper and inconsistent with the evaluation criteria outlined in the solicitation. 
Under the most heavily weighted criterion Qualifications of the Firm, JW asserted that it 
had two recent timber sale environmental impact statements and was working on a third 
at the time of evaluation. JW also maintained that NRMC had no experience regarding 
environmental impact statements. JW concluded that its score for firm qualifications 
should have been greater that NRMC’s score. 

The agency stated that the evaluation was fairly performed and in keeping with 
the provisions in the RFP. In response to specific assertions by the protester, the Forest 
Service only responded by repeating the evaluation criteria and the method for averaging 


scores. GAO found that the technical evaluation was not adequately supported. The 
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Comptroller General continued stating that without adequate supporting documentation 
for the evaluation, a proper award determination was not possible. Furthermore, the 
contracting officer debriefing letter stated plainly that price was the basis for the agency’s 
award decision. The RFP stated that price was last in importance and only the deciding 
factor if two proposals were found to be technically equal. GAO held that the record did 
not contain sufficient contemporaneous documentation, information, or analysis to make 
a decision of technical equality. GAO stated that the FAR 15.612(d)(2) (pre-rewrite) 
required that the documentation supporting selection decisions show the relative 
differences among proposals; their strengths, weaknesses, and risks; and the basis for 
award decisions. GAO held that the Forest Service did not adhere to the regulation and 
sustained the protest. The Comptroller General recommended discussions (if deemed 
necessary by the agency), revised BAFOs, and reevaluation. [Ref. 18] 
E. ANALYSIS AND CONCLUSION 

The purpose of this analysis is to give the contracting officer an indication of the 
common pitfalls when evaluating technical merit in best value contracts. 

1. Craft An Accurate Solicitation That Reflects the True Agency 

Requirement 

A best value source selection can be undermined early in the contracting process 

by way of a sloppily crafted solicitation. HG is a prime example where the true desires of 


an agency are not reflected in the RFP. The Forest Service stated during the protest that a 
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conceptual drawing of the supervisor’s office met the minimum needs of the agency. 
Contrary to the drawing, the solicitation stated in writing that the computer room must be 
away from microwave and radio transmitting equipment. These contradictory 
requirements resulted in the agency essentially ignoring their written (overriding) 
requirement when making their award decision. The Comptroller General analysis and 
decision were obvious in this case. GAO held that the agency and awardee did not follow 
the solicitation. This holding stems from a previous protest decision (Southwest Marine, 
Inc., American Systenis Engineering Corporation; B-265865.3, B-265865.4; 23 January 
1996) where GAO asserted that the evaluation of proposals must be reasonable and bear a 
rational relationship to the announced criteria upon which competing offers are to be 
selected. [Ref. 20] The implication for the contracting officer is that great care must be 
taken to ensure the solicitation is well written and reflects the actual customer 
requirement. Poorly crafted solicitations are susceptible to protest and sustainment. 

2 Evaluate Proposals in Accordance with the Solicitation 

The inability or unwillingness to follow the evaluation criteria in the solicitation 1s 
a prevalent miscue by Federal agencies. 757 documents an improper technical evaluation 
because a Federal agency chose not to follow the solicitation. Technology Services 
protested the award of a Hawaiian grounds maintenance contract in which the awardee 
was not held to the requirements of the RFP by the agency. The RFP required a work 


scheduling system for which the awardee issued only a promise to develop. The agency 
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(Air Force) accepted the promise and changed the awardee’s score from the lowest 
possible (Unacceptable) to the highest attainable (Exceptional) rating. However, the 
good will and trust of the Air Force were not enough to convince the Comptroller 
General. The protester successfully argued that the awardee simply had not met the 
requirements of the RFP. GAO agreed. Rather than the typical recommendation of 
reevaluation, GAO held that the awardee’s proposal was unacceptable and recommended 
termination and immediate award to the protester. This is despite the fact that the 
protester’s proposed price was nearly $2 million greater that the proposed price of the 
awardee. This holding stemmed from a previous protest decision (Tidewater Homes 
Reality, Inc., B-274689, 26 December 1996) where the Comptroller General held that an 
agency evaluation may not lack a reasonable basis or conflict with the stated evaluation 
criteria for award. [Ref. 22] The implication for contracting officers is that they must 
adhere to the evaluation criteria set forth in the solicitation when measuring technical 
merit. They must also thoroughly research solicitations to ensure they are free from 
contradictions and ambiguities. The failure to do so can result in unfair and unreasonable 
contract award or perhaps the purchase of an undesirable product or service by an agency. 

3. Apply Supporting Narratives to Evaluation Records 

The requirement to attach narratives to the previously required score sheets and 
rankings is a product of the FAR Part 15 rewrite. However, GAO stressed the need for 


supporting documentation in three pre-rewrite protests. 
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MECSA is a protest that challenged a Federal agency award of a contract for the 
cleaning and painting of four miter gates in the Panama Canal. The agency’s technical 
evaluation was undermined by the lack of contemporaneous supporting documentation. 
The agency had no justification in the evaluation record for overriding the protester’s 
lower price and awarding to Formal Management Systems (FMS). In addition, the lack 
of supporting statements called into question whether the agency actually accounted for a 
special certification (QP-2) held by one of the protester’s subcontractors. The agency 
maintained that it had given the subcontractor certification proper consideration. They 
may have, but the record did not support their assertion. As a result, GAO found that the 
protester had been prejudiced and sustained the protest. 

Jones is a protest that reaffirms the crippling need for narrative in best value 
source selection. J. A. Jones protested the Army Corps of Engineers award of a contract 
for base repair and construction. The agency had only score sheets with bullet statements 
to support their evaluation. When their evaluation was found to be unreasonable and 
inaccurate, they claimed that the bullets were only impressions and not the full 
explanation as to why they selected Intersteel Inc. as the awardee. GAO held that if the 
bullets did not represent their rationale for selecting Intersteel, then their level of 
documentation decreased from little to none. The Army had committed the double error 
of improperly evaluating the proposals, and providing little or no documentation to 


justify their selection. GAO termed the amount of supporting documentation as a “dearth 
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of evaluation narratives.” The Corps of Engineers left GAO with little choice but to 
sustain the protest and recommend reevaluation. 

JW is a third protest where lack of supporting documentation resulted in 
sustainment. The agency in this case clearly did not meet the FAR Part 15 pre-rewrite 
standard requiring strengths, weaknesses, risks, and basis for award to be included in the 
evaluation record. The agency (Forest Service) appeared to have improperly evaluated 
both the protester and awardee in the Quality of Firms factor. The protester had two 
complete and one in progress environmental impact statements while the awardee had 
none. Still, the awardee received a higher score in this heavily weighted factor. To make 
matters worse, the only supporting documentation consisted of a one paragraph per 
offeror narrative that in no way explained the scoring of the factor in question or the 
specific reasons for award selection. GAO sustained the protest. 

MECSA, Jones, and JW demonstrate that the nature of tradeoff is not always clear 
and sometimes tenuous. The use of narrative comments provides the agency with an 
opportunity to explain fully their reasons for scoring, ranking, and source selection. The 
requirement for narrative comments (in addition to strengths, weaknesses, risks, and basis 
for award) was not in force when these contracts were awarded, yet GAO appeared 
compelled to send the message that narratives are important. The implication for 
contracting officers is that they must now make liberal use of narrative because the FAR 


Part 15 rewrite mandates it. More importantly, it allows a Federal agency to announce its 
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methods, motives, and reasons for tradeoff and source selection. The key fact is that 
contracting officers will be well served if they supply narratives with best value awards 
that include the agency’s very thought processes in arriving at a selection decision. 
Failure to convey the rationale for tradeoff and source selection adequately could be 
perceived by GAO as unreasonable technical evaluation and source selection. 

4. Hold Meaningful Discussions Regarding Technical Evaluation When 

Required 

IDS is a protest that could have been avoided through the use of meaningful 
discussions. Discussions both before and after the FAR Part 15 rewrite involve essential 
information for determining the acceptability of a proposal or provide the offeror an 
opportunity to revise the proposal. The protester maintained that its typographical error 
in delivery schedule (30-45 days instead of the required 15) could have been rectified 
through discussions. The RFP stated discussions would not be held, and the agency 
believed all dialog up to that point had been clarifications. GAO held that the 
clarifications were actually discussions and sustained the protests. This decision was 
based in the FAR 15.601 (pre-rewrite) statement that defines discussions as exchanges 
with offerors that involve key information for determining the acceptability of a proposal. 
In addition, all the clarifications qualified as discussions because BAFOs were issued and 
accepted as a result, so the offerors were provided an opportunity to modify their 


proposals. The presence of discussions then meant that they had to be meaningful. GAO 


7] 


held that the discussions with the protester were not meaningful because they did not 
address aspects of an offeror’s proposal that must be corrected for the offeror to have a 
reasonable chance to obtain award. The agency failed to address the delivery schedule; 
therefore, the discussions were not meaningful. The implication for co officers 
1s that all exchanges with offerors must be scrutinized and identified (e.g. discussions, 
clarifications). If discussions are held, they must be meaningful and conducted with all 
offerors in the competitive range. The Part 15 rewrite has opened the door for increased 
dialog through communications during competitive range determination and for adverse 
past performance references. This forces contracting officers to be even more aware of 
the consequences of their exchanges with offerors. 

5. Screen all Offeror Documents for Relevant Technical Information 

Cygnus is a protest that should never have happened. Cygnus Corporation 
protested the award of a contract operation and management of the National Adoption 
Information Clearinghouse by the Department of Health and Human Services. Here, the 
agency apparently overlooked or forgot to include information (potential contract 
problems and solutions) from the protester’s BAFO in the technical scoring. The record 
contained no indication that the information had been received, evaluated, or included on 
the scoring sheets. GAO found the technical evaluation unreasonable and sustained the 


protest. The lesson for contracting officers is simple. Carefully review all documents 
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from offerors for pertinent information. Failure to do so could result in an embarrassing 
protest for which a Federal agency has no defense. 

Contracting officers can learn a great deal from this chapter. First, crafting an 
accurate and clear solicitation is vital. Second, following the solicitation is critically 
important. Third, narratives in conjunction with other supporting documentation can 
explain an agency’s rationale for technical tradeoff and best value source selection. 
Fourth, meaningful discussions about a technical evaluation may be necessary to allow an 
offeror to correct a proposal and have a reasonable chance to obtain award. Finally, all 
offeror correspondence must be carefully reviewed for relevant technical information that 


could affect the outcome of the source selection decision. 





VI. COST/PRICE 

A. INTRODUCTION 

Cost or price is one of the most important tradeoff elements regarding best value. 
The reason is simple. Cost/Price must be evaluated in all proposal evaluations as a matter 
of regulation. Cost/Price is especially important in best value contracting because it is the 
single element against which all other elements (past performance, technical merit, etc.) 
are traded to obtain best value. The six sustained protests in which improper evaluation 
of price or cost was a sustaining argument are: The Arora Group, Inc., Barents Group, 
L.L.C.; Boeing Sikorsky Aircraft Support, Geo-Centers, Inc.; Sylvest Management 
Systems Corporation; and Tri-State Government Services, Inc. These protests are divided 
into three categories: 

1. Agency failure to evaluate the total proposal cost. 

2. Improper agency evaluation of cost realism and/or price realism. 

3. Agency failure to follow the solicitation. 
Three of the protests were also sustained for improper agency discussions regarding the 
evaluation of price or cost. This chapter discusses the affect of the FAR Part 15 rewrite 
on cost or price evaluation, describes the six sustained protests, and provides analysis 


about Federal agency miscues when evaluating cost or price. 
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B. COST/PRICE AND FAR PART 15 

The Part 15 rewrite does not make any significant changes regarding price or cost. 
As previously stated, cost/price is required to be evaluated in every source selection. 
Cost/price is often analyzed to ensure a fair and reasonable price for the Government. 
Competition normally establishes price reasonableness. With fixed-price contracts, 
comparison of the proposed prices usually satisfies the requirement for price analyses. 
Cost-reimbursement proposal evaluations must include cost realism analyses to determine 
what the Government should expect to pay for proposed goods or services. [Ref. 1: 
15.304, 15.305] The key behind the rewrite is the increased emphasis on best value 
contracting. The result is a different and sometimes more difficult handling of price or 
cost in proposal evaluation. Before the widespread use of best value contracting, most 
contracts were awarded by virtue of being either lowest price-technically acceptable or 
highest technically rated. Best value pushes the contracting officer within this spectrum 
and cost or price must always be a tradeoff element. The contracting officer must 
exercise good judgment in trading off non-cost factors (technical merit, past performance, 
labor qualifications, etc.) against price or cost. Increased best value awards mean more 
complexity in the contracting officer’s proposal evaluations. Cost/price is a key issue for 


the contracting officer to manage on every proposal evaluation. 
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Cc: AGENCY FAILURE TO EVALUATE THE TOTAL PROPOSED COST 

OR PRICE 

1. Boeing Sikorsky Aircraft Support 

Boeing Sikorsky Aircraft Support; B-277263.2, B-27263.3; Comptroller General 
of the United States; 29 September 1997, (Boeing) is a protest in which agency failure to 
evaluate the total proposed cost, improper agency evaluation of labor cost, and agency 
failure to hold meaningful discussions resulted in sustainment. Boeing protested the U.S. 
Special Operations Command (SOCOM) award of a cost-plus-award-fee contract for the 
operations and maintenance of the Special Operations Forces Support Activity (SOFSA) 
at depots in Richmond and Lexington, Kentucky. SOFSA consists of Government- 
owned, contractor-operated facilities that provide logistics support for the Special 
Operations Forces (SOF) including equipment repair, modification, and sustainment; 
prototype and low volume manufacturing; maintenance management; and life cycle 
support. The contract was for one base year with four, one-year options. SOCOM 
determined Raytheon E-Systems (Raytheon) to be the offeror whose proposal represented 
the best value to the Government. 

The RFP was written to allow offerors as much latitude as possible to propose 
innovative ways of doing business and to encourage creativity in problem solving. 
Offerors were required to complete seven sample tasks. Six of the tasks were outlined in 


the RFP, and the seventh was to be performed within two weeks of the closing date to 
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simulate the rapid reaction environment of SOFSA. The proposals were evaluated in the 
following areas (descending order of importance): (1) technical, (2) management, (3) 
performance risk, and (4) cost. The technical area was comprised of the seven required 
tasks where management and performance risk each had various sub-factors for 
evaluation. Cost evaluation encompassed five sub-factors: (1) cost estimating and 
control, (2) quality control (3) security management, (4) schedule planning and control, 
and (5) management effectiveness. The cost evaluation scheme used a risk scale (low, 
moderate, high) to score the various sub-factors. This cost evaluation structure was 
designed to measure each offeror’s ability to estimate accurately the cost of completing 
logistics support tasks, as well as each offeror’s overall proposal cost. Five offers were 
received by SOCOM. SOCOM found Raytheon to be equal to Boeing in technical merit 
and performance risk and superior to Boeing in the management and cost factors. 
SOCOM awarded the contract to Raytheon. Boeing Sikorsky protested. 

The protester raised two challenges to SOCOM’s cost evaluation. First, the 
protester argued that the agency failed to take into account the most probable cost of the 
proposals. In other words, SOCOM evaluated cost only from a risk perspective—not 
total cost difference between competing proposals. The agency countered by stating that 
it was not required to consider the most probable costs because the solicitation made no 
such provision. GAO showed that the RFP stated that the total dollar figure would be 


used by the Government as an indicator of total cost for the expected term of the contract. 
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Even if the solicitation supported the agency, GAO found SOCOM’;s assertion that 
consideration of total cost was not a requirement to be legally incorrect. GAO stated that 
CICA (1984) mandated inclusion of cost or price as a significant factor in the evaluation 
of proposals. GAO determined that SOCOM failed to give significant consideration to 
cost and found their evaluation to be unreasonable. 

The protester’s second argument maintained that SOCOM erroneously increased 
its labor costs. The protester had failed to consolidate all labor costs into one figure in its 
proposal in accordance with the RFP. Even though the summation of labor cost was not 
in the proposal, there were no missing labor hours in the offer. As a result, the agency 
erroneously adjusted Boeing’s cost upward to compensate for what appeared to be 
missing labor hours. GAO found that the accuracy of the labor hour assessment was not 
the critical issue. The key issue regarding the labor hours was that SOCOM failed to 
conduct meaningful discussions with Boeing regarding a perceived significant error in 
their proposal. Boeing should have been placed on notice regarding this perceived 
weakness in their proposal. The resultant cost adjustment was significant, for Boeing had 
been assigned a high risk rating in overhead and labor cost evaluations. Moreover, the 
elimination of the cost adjustment through discussions would have resulted in Boeing 
having a lower cost proposal than Raytheon. 

The lack of discussions was the linchpin in this protest. The failure to evaluate 


the total cost would not normally have been crucial in this particular award because 
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Raytheon had a proposal with an overall advantage in non-cost factors as well as cost 
factors. However, with the agency error regarding the assessment and subsequent upward 
adjustment to labor hours, Boeing may well have been denied cost advantage due to the 
evaluation. A Boeing cost advantage coupled with a Raytheon non-cost advantage would 
have required the agency to conduct a tradeoff analysis to see which proposal represented 
the best value to the Government. In this case, no such tradeoff analysis was executed. 
GAO found that Boeing had been prejudiced by the combination of agency failure to 
compare total costs, agency error in evaluating labor cost, and agency failure to hold 
meaningful discussions. The Comptroller General sustained the protest. GAO 
recommended that SOCOM reopen negotiations with all offerors, request revised 
proposals, and reevaluate the proposals for a best value award. [Ref. 24] 

2: Sylvest Management Systems Corporation 

Sylvest Management Systems Corporation; B-275935, B-2753935.2; Comptroller 
General of the United States; 21 April 1997, (Sylvest) is a protest where agency failure to 
evaluate the total price (all costs) resulted in sustainment. Sylvest protested the 
Tennessee Valley Authority (TVA) award of a contract for the purchase of computer 
hardware and maintenance support services. TVA determined BTG Inc. (BTG) to be the 
firm whose proposal represented the greatest value to the Government. 

The RFP described three schedules of required services. Schedule I included the 


purchase of computer maintenance services. Schedule II and III were for the purchase of 
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new hardware plus various hardware and software upgrades to TVA’s current computer 
system. The solicitation stated that multiple contracts could be issued to satisfy the 
requirements contained within the three schedules. This protest only addressed the 
TVA’s award under Schedule III for new hardware. Under Schedule III, each offeror was 
required to submit offers for nine servers of three sizes (small, medium, and large) from 
three separate manufacturers. The offerors were required to test and certify that the 
hardware met both industry standards and TVA-specific benchmarks. The RFP stated 
that the contract would be awarded to the offeror who received the highest cumulative 
score for both technical and price evaluation factors. TVA evaluated all proposals, held 
discussions, received two rounds of BAFOs, and awarded a contract under Schedule III to 
BTG. Sylvest Management Systems protested. 

The protester argued that the agency erroneously evaluated BTG’s price proposal. 
The protester asserted that the BTG did not propose the same hardware configuration that 
they used to test to TVA benchmark standards successfully. The protester also stated that 
a significant amount of memory had to be added to BTG’s basic (proposed) hardware 
configuration to meet TVA benchmarks successfully. The protester pointed out that BTG 
had listed the extra hard disk memory as an option in the proposal, and as a result, the 
agency did not consider all costs (cost of required extra memory) when determining best 


value. 
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TVA countered stating that the certification in BTG’s BAFO was a satisfactory 
indication that BTG was offering the optional additional memory as part of its basic 
package. GAO described TVA’s position as untenable. The Comptroller General found 
(without yet evaluating the certification) that it was clear from BTG’s BAF O that the 
extended price did not include the extra hard disk memory required to satisfy TVA 
benchmarks. GAO further held that the certification did not incorporate the required 
additional memory in the offered system at no additional cost. BTG’s certification stated 
only that their proposal had been updated to reflect the benchmarked configuration. The 
awardee’s certification did not state that all items included in the required configurations 
were included in the systems prices as required by the solicitation. The Comptroller 
General also found that BTG’s exclusion of the substantial cost required to upgrade the 
hard disk resulted in BTG having a lower cost than Sylvest. GAO found that Sylvest had 
been prejudiced and recommended reevaluation of the BAFOs submitted in response to 
Schedule III. [Ref. 26] 

D. IMPROPER AGENCY EVALUATION OF COST REALISM AND/OR 

PRICE REALISM 

1. Barents Group L.L.C 

Barents Group, L.L.C.; B-276082, B276082.2; Comptroller General of the United 
States; 9 May 1997, (Barents) is a protest where improper agency cost realism analysis 


failed to detect awardee conditions placed on RFP mandated pricing multipliers. The 
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result was sustainment. Barents protested the award of an Agency for International 
Development (AID) contract for technical expertise in: (1) economic and institutional 
analysis and (2) private sector development issues. The solicitation envisioned between 
four and six indefinite quantity contracts. Each of these contracts was to be for three 
base-years and two option years with one contract set aside under the Small Business 
Administration’s 8(a) program. AID found Chemonics International, Inc. (Chemonics); 
Booz-Allen & Hamilton, Inc. (Booz-Allen); Abt Associates, Inc. (Abt); Carana 
Corporation (Carana); and Deloitte Touche Tohmatsu (Deloitte) to be the offerors who 
submitted proposals representing the greatest value to the Government. 

The RFP outlined the award of contracts in two functional areas; however, 
protests were issued regarding only line item 0001—privatization issues. The solicitation 
stated that awards would be made to the responsible offerors submitting acceptable, 
reasonably priced proposals that offered best value to the Government. Each proposal 
was to be assigned a score for technical and cost. Technical was weighted 60 percent and 
cost 40 percent. Technical factors included the following (25 points each): (1) personnel 
qualifications and experience, (2) quality and responsiveness, (3) demonstrated corporate 
experience, and (4) past performance. 

Under cost, the solicitation called for offerors to propose a maximum fixed daily 
salary for seven labor categories covering the base and option periods. In addition, the 


offerors were supposed to issue two fixed multipliers, one for U.S. expatriate staff and the 
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other for non-U.S. personnel. The multipliers were to contain all payroll costs, indirect 
costs, home/corporate office secretarial/administrative support, computer rental, report 
preparation costs, and profit or fee. The solicitation stated that the agency would 
calculate an average burdened daily rate using the multipliers and the agrees fixed 
daily salaries. 

AID received 17 offers. Agency officials then scored the offers under the four 
technical criteria, applied the 60-40 technical to cost factor, and ranked the offerors as 
follows: Chemonics, Deloitte, Carana, Booz-Allen, Abt, and Barents. Barents was the 
only offeror in the top six that did not receive a contract award. The agency said the 
awards were not based solely on scores. A best value analysis was performed. The 
agency stated that it initially decided to make awards to Chemonics, Deloitte, and Carana 
because they were in the top six technically and in the top four (lowest) in price. After 
the first three firms were ranked, the contracting officer ranked Booz-Allen fourth overall] 
with Abt finishing fifth—ahead of Barents. The agency noted that while Barents 
technical score exceeded Booz-Allen’s and Abt’s, the lower prices of Booz-Allen and Abt 
offset any technical advantage enjoyed by the protester. The agency awarded contracts to 
the five firms ranked higher than Barents plus one SBA 8(a) company. Barents protested. 

Among other allegations, Barents argued that AID failed to perform a cost realism 
analysis in accordance with the solicitation. Barents maintained that AID was required to 


evaluate whether the costs proposed by each offeror were consistent with the firm’s 
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technical proposal. The RFP also stated that costs could have been adjusted as a result of 
a cost realism analysis to evaluate the proposals fairly. Specifically, Barents asserted that 
Booz-Allen placed a contingency in their proposal, which should have been found 
unacceptable by the agency. Booz-Allen’s proposal stated: “Some of the labor hours 
included in this cost estimate are bid at off-site rates, substantially lower than rates 
normally applicable to services performed at our facility. These rates are based upon the 
provision of the following property/facilities by the client or one of our subcontractors. 
(a) Office facilities (b) Communications, 1.e. local and long distance telephone service 
(c) Copying facilities (d) Computer facilities, as required (e) General office supplies 

(f) parking facilities. The costs proposed are specifically conditioned upon the 
availability of the items set forth above.” [Ref. 23] Barents argued that this provision 
made the fixed daily salaries and multipliers contingent on the availability of the listed 
items. Therefore, Booz-Allen failed to comply with the RFP because the listed items 
were supposed to be factored into the salaries and multipliers. Barents also maintained 
that each of Booz-Allen’s proposed multipliers for personnel included the statement: 
“Assumes bilateral agreements between [AID] and participating government cover any 
cost incurred in respect to tax, duties, bonding, and any social welfare costs.” [Ref. 23] 
Barents stated that the solicitation did not outline such agreements. The protester argued 
that since Booz-Allen’s daily salaries were based on the assumption that costs would be 


paid by AID or the participating Government, Booz-Allen’s proposal was unacceptable. 
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AID countered by stating only that Booz-Allen’s proposal was in conformity with 
the solicitation. However, the agency had no explanation for the awardee’s statement that 
the proposed costs were specifically conditioned upon the availability of the listed items 
that were supposed to already be included in the fixed daily salaries nd multipliers. AID 
also took the position that none of the alleged objectionable language from Booz-Allen’s 
proposal was included in the final contract, so it had no impact on the award. GAO found 
the agency evaluation unreasonable and stated that it was clear that Booz-Allen’s 
proposal did not comply with the solicitation requirement for fixed multipliers. In 
conjunction, AID’s cost realism analysis of the awardee’s proposal was inadequate. The 
Comptroller General also addressed AID’s assertion that the contract was devoid of 
objectionable language; thus, the proposal flaws were of no matter. GAO stated that an 
awarded contract does not have to incorporate all aspects of a proposal. However, the 
contract may not vary materially from the offer. GAO found that the contract was 
materially different. In addition, GAO held that the conditions placed in the proposal by 
Booz-Allen removed the fixed nature (cap) of the prices. The awardee circumvented the 
cap in this case, and the Government was no longer shielded from cost growth. In such a 
situation, GAO held that a proper cost realism analysis was required. GAO found that a 
proper cost realism analysis was not held because it did not include an evaluation of the 
extent to which Booz-Allen’s proposed costs--represented what the contract should cost. 


GAO recommended that AID clarify the RFP, request revised BAFOs, and reevaluate. 
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2, Geo-Centers, Inc. 

Geo-Centers, Inc., B-276033, Comptroller General of the United States, 5 May 
1997, (Geo) is a protest in which agency failure to properly conduct price and cost 
realism analyses on sample task costs resulted in sustainment. Geo protested the Army’s 
award of a cost-plus-fixed-fee, task order requirements contract for scientific and 
technical support services for the Health Effects Research Program (HERP). HERP was 
part of the Army’s Center for Health Promotion and Preventative Medicine at Aberdeen 
Proving Ground, Maryland. The contract was for one base year with two option years. 
The Army determined Dynamic Corporation (Dynamic) to be the offeror whose proposal 
represented the best value to the Government. 

The RFP outlined five evaluation factors: technical merit, management merit, 
performance risk, cost, and subcontracting plan. The technical merit and management 
merit were to be combined and scored numerically on a 1,000 point scale. Performance 
risk was to be evaluated on a narrative basis. The subcontracting plan was to be graded 
on a go/no go (pass or fail) basis. The solicitation stated that technical merit and 
management merit were significantly more important than cost, and cost was slightly 
more important that performance risk. Cost was to assume greater importance in the 
event of two or more proposals being evaluated as equal in the technical merit and 
management merit criteria. The RFP contained three sample tasks for which each offeror 


was to submit technical responses with sample task cost proposals. These sample task 
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cost proposals were in addition to the total offeror cost proposal required by the 
solicitation. 

The sample task cost proposals were to be evaluated for price reasonableness and 
cost realism, and the solicitation defined both terms. Cost realism was ehe as 
“whether or not the offeror has proposed sufficient resources to successfully perform the 
contract and sample task work.” Price realism was outlined as “a matter of the 
competitiveness of the offeror’s proposal, considering, the price and the merit factors 
area.” [Ref. 25] Geo received initial proposal, post discussion, and BAFO scores of 930, 
998, and 1000 respectively (1000 point scale). Dynamic received scores of 680, 775, and 
1000. Both proposals were found to be technically equal, so the Army awarded the 
contract to the offeror with the lowest overall proposed cost—Dynamic. Geo protested. 

Geo argued that the agency failed to conduct cost realism or price realism 
evaluations on the sample task costs in Dynamic’s BAFO. The protester’s argument 
pointed to the fact that Dynamic increased labor hours and costs for the three sample 
tasks, yet they lowered their overall proposal costs by 25 percent. Geo argued that these 
major cost shifts should have drawn reevaluations of price and cost reasonableness for the 
sample tasks from the Army. The agency maintained that the pricing in the sample task 
was not intended as a tool for establishing a basis for price reasonableness. The RFP 


supported the agency’s assertion stating that the sample tasks were not used to determine 
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the total contract price; rather, they were merely a gauge for contractor understanding of 
the effort required for contract completion. 

However, GAO stated that by using a task order requirements contract, the agency 
acknowledged that it was not able to determine the exact quantity of services and 
materials needed during the performance period. Therefore, the cost to the Government 
would have varied based on the actual task orders issued, and most likely, with a 
contractor’s efficiency in performing the issued tasks. The offerors’ overall proposal 
costs were not dependable. GAO stated that the sample task cost proposals were a 
significant indicator of future cost to the Government. In addition, the solicitation 
advised that sample task costs would be evaluated for both cost realism and price 
reasonableness, and the agency was bound to use the evaluation tools identified in the 
RFP. GAO determined that the Army failed to conduct price and cost realism evaluations 
on Dynamic’s BAFO. The failure to analyze the cost realism of the proposed sample 
tasks left the agency open to the possibility that Dynamic did not accurately account for 
all of the costs that will be incurred during the performance. GAO further determined 
that the Army failed to conduct any price realism analyses of sample task costs (in 
proposals or BAFQOs) despite the requirement in the solicitation. GAO found Geo had 
been prejudiced because the Comptroller General could not conclude that the protester 


would not have had a substantial chance of award had the agency properly analyzed the 
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sample task costs. As a result, GAO sustained the protest and recommended 
reevaluation. [Ref. 25] 

3: Arora Group, Inc. 

Matter of Arora Group, Inc., B-277674, Comptroller General of the United 
States, 10 November 1997, (Arora) is a protest in which the Federal agency unreasonably 
determined that the protester’s proposal lacked price realism. Arora Group protested the 
U.S. Navy’s award of a fixed-price contract for the services of ten pharmacists for the 
National Naval Medical Center in Bethesda, Maryland. The contract was to be a small 
business set aside under the SBA’s 8(a) program. The solicitation sought the services of 
ten full-time and five optional pharmacists for a base period of four option periods 
spanning a maximum of five years. The Navy determined Saratoga Medical Center, Inc. 
(SMC) to be the best value offeror. 

The RFP stated that the best value award would be made to the offeror whose 
proposal conformed to the specified minimum healthcare worker qualifications and 
offered the Government with the best combination of past performance and price. 
Offerors had to provide proof that they would provide at least ten individual healthcare 
workers. Past performance was outlined as significantly more important than price. In 
the event two or more offerors had equal past performance, price would be the deciding 


factor. The solicitation also stated that price realism analyses would be conducted to 
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preclude unrealistically low cost estimates. These analyses were to be conducted within 
the context of the offerors’ past performance. 

The agency received eight proposals. Both the protester and awardee earned past 
performance grades of Good. With respect to price realism, the Federal Acquisition 
Circular (FAC 90-23) recommended an annual wage escalation rate of three percent. The 
Navy had indicated that provisions for wage escalation should be included in each 
offeror’s proposal to guard against labor turnover and wage increase. The protester failed 
to propose wage escalation during the option periods covered in the RFP. The Navy 
determined that the protester’s failure to provide for any salary escalation presented a 
price realism issue because it created risk that the offeror might not be able to maintain a 
pool of qualified personnel or recruit suitable replacements. The protester was advised of 
this perceived shortcoming in its proposal through discussions but did not submit wage 
escalation data in its revised proposal. The agency determined SMC to be the best value 
offeror and awarded them the contract. Arora Group protested. 

The protester argued that the Navy’s price realism analysis was not in accordance 
with the RFP and plainly erroneous. The protester asserted that the agency incorrectly 
conducted price realism analysis because they failed to consider the fact that Arora Group 
addressed the recruitment/retention issue through long-term employment agreements. In 
addition, the protester had planned for recruitment costs in its General and Administrative 


(G&A) budget in the event of personnel turnover. The Navy countered that the protester’s 
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plan did not provide convincing evidence that they would be able to retain the healthcare 
workers through the five-year span of the contract without wage escalation. The agency 
also stated that the protester’s G&A budget for recruitment was vague. 

GAO held that the agency’s decision to reject Arora’s proposal because its failure 
to propose wage escalation created an unacceptable price realism risk was neither 
reasonable nor consistent with the evaluation criteria. The Comptroller General stated 
that the agency failed to adequately evaluate the information presented by the protester. 
With respect to the employment contracts, the Navy had requested copies of the labor 
agreements during discussions. The protester failed to provide these agreements; 
therefore, the Navy essentially ignored the possible impact of this approach. GAO found 
this action to be unreasonable. GAO stated that while copies of labor agreements may 
have provided reassurance to the Navy, Arora’s failure to provide the copies did not mean 
the agency could completely discount the protester’s efforts in this regard. Furthermore, 
the agency improperly evaluated the protester’s G&A budget for recruitment and 
understated its value by 90 percent. GAO held that the combination of the unreasonable 
discounting of the labor agreements and the erroneous evaluation of the G&A budget 


prejudiced the protester. The Comptroller General sustained the protest. [Ref. 32] 
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E. AGENCY FAILURE TO FOLLOW THE SOLICITATION 

1. Arora Group, Inc. 

Arora also demonstrated agency failure to follow the solicitation. The RFP stated 
that the price realism analyses were to be conducted within the context of the offerors” 
past performance. Both the protester and awardee received past performance scores of 
Good. The protester had offered a lower price than the awardee. However, the agency 
ignored the evaluation criteria and rejected Arora Group”s proposal due to concern about 
price realism (wage escalation) risk. With respect to the protester’s proposal, GAO held 
that the Navy failed to conduct price realism analysis and tradeoff as outlined in the RFP. 
The agency had improperly made price realism a separate and determinative evaluation 
factor. GAO sustained the protest and recommended reconsideration of the evaluation 
criteria, BAFOs, and reevaluation. [Ref. 32] 

2: Tri-State Government Services, Inc. 

Tri-State Government Services, Inc., B-277315.2, Comptroller General of the 
United States, 15 October 1997, (Tri-State) is a protest in which agency failure to hold 
the awardee to the terms ofthe RFP combined with improper post-BAFO discussions 
resulted in sustainment. Tri-State protested the Defense Logistics Agency (DLA) award 
of a fixed firm price contract for the removal, transportation, and disposal of hazardous 
waste items located at various Defense Reutilization and Marketing Sites (DRMS). The 


solicitation anticipated an 18-month base period with two option years. DLA found 
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Associated Environmental Services, Inc. (AES) to be the offeror whose proposal 
represented the best value to the Government. 

The RFP required separate technical, price, and past performance proposals and 
warned offerors that proposals, which did not include all required information in the 
prescribed format, could be excluded. The solicitation outlined price, past performance, 
socioeconomic plan, and Mentoring Business Agreement (MBA) participation as 
evaluation factors. Past performance and price were outlined as equal in importance; 
furthermore, they were also stated to be significantly more important than the value of 
socioeconomic plan and MBA combined. The RFP divided the work to be performed in 
to 94 contract line items (CLIN) for different waste substances. Each proposal was to 
provide a single unit price for each CLIN in a space provided in the solicitation price 
schedule. The single unit prices were to be multiplied by the agency’s estimated quantity 
(of hazardous waste) to arrive at an extended price for each CLIN, and then all CLIN 
extended prices would be summed to reveal a total price. Eleven offerors submitted 
proposals with only five surviving the competitive range determination. Four of the five 
proposals (including AES and Tri-State) were graded as good under both past 
performance and a best value composite rating. Price became the determining factor. 
AES ranked first in price with a total of $ 3.29 million. Tri-state finished second in price 
with an offer totaling $ 3.44 million. AES was awarded the contract and Tri-state 


protested. 
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Tri-state argued that AES improperly modified the agency’s price schedule and 
should not have been considered. Instead of submitting single unit prices for each CLIN, 
AES split DLA’s estimated quantities listed on the price schedule and submitted one 
price for a number that it described as first quantities ordered and a different price for 
what it designated as next quantities ordered. In each of the split CLINs, the prices were 
higher for the first quantities ordered and lower for the next quantities. The protester 
contended that the modified pricing scheme gave AES an unfair advantage because such 
a pricing method was not available to all offerors. Tri-state also asserted that DLA 
conducted improper post-BAFO discussions with AES. Tri-state maintained that the 
agency re-opened discussion with the awardee to negotiate prices for any quantities 
ordered beyond the Government estimates shown in the RFP. 

Regarding the pricing scheme, the agency countered claiming that the RFP did not 
prohibit AES’s pricing strategy. DLA considered the awardee’s pricing strategy as only a 
minor irregularity and consistent with the RFP. GAO held that the agency improperly 
accepted the awardee’s offer because it did not conform to the material terms and 
conditions of the solicitation. GAO stated that an irregularity in an offer that benefits the 
offeror must be extended to all firms submitting proposals. GAO continued by stating 
that if the beneficial irregularity is not extended to all offerors, the proposal containing 


such an irregularity may not be accepted. In determining prejudice, GAO found that if 
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Tri-State had been allowed to use AES’s pricing method, the protester would have had a 
lower overall price than AES by $ 130,000. 

With respect to Tri-State’s assertion of improper discussions with the awardee, the 
agency asserted that the contracting officer believed that it was reasonably clear from 
their proposal that the prices AES had quoted for next quantities would apply to future 
quantities beyond the Government estimates. DLI claimed that they only conducted 
clarifications with AES to clear the ambiguity as to the price of these follow-on 
quantities. GAO held that the exchanges were actually discussions for two reasons. 
First, the Comptroller General held that it was not reasonably clear that the next quantity 
prices would apply beyond the Government estimates. Second, the information 
exchanged was necessary to determine the acceptability of AES’s proposal. GAO held 
that Tri-State was prejudiced because of the improper pricing and discussions. GAO 
sustained the protest with a recommendation to terminate the AES contract and award to 
Tn-State. [Ref. 27] 

F. ANALYSIS AND CONCLUSION 

The purpose of this analysis is to provide contracting officers with insight as to 
possible agency miscues when evaluating cost or price. 

1. Evaluate the Total Price or Cost 

Failure to evaluate the total price or cost is an agency error that must not be 


committed. However, Boeing showed that agency efforts to evaluate cost risk cannot be 
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executed to the exclusion of a total cost comparison between offerors. The agency 
(SOCOM) seemed to be unfamiliar with this requirement even though their RFP clearly 
mandated a tota] cost comparison. GAO cited a general rule of law originating with 
CICA (1984) that required inclusion of cost or price as a significant factor in the 
evaluation of proposals. The implication for contracting officers is clear. Failure to 
consider total cost 1s a very winnable protest for firms. Contracting officers cannot get so 
caught up with slick cost analysis tools that they make tradeoffs without consideration of 
a basic tota] costíprice comparison. Boeing also demonstrated an erroneous agency 
adjustment of labor cost coupled with a failure to inform the offeror through meaningful 
discussions can contribute to a sustainment. Although the protester did not provide a 
summation of its labor cost figures as was required by the RFP, SOCOM clearly erred by 
(1) failing to discuss the apparent mistake with the offeror and (2) erroneously adjusting 
the Jabor cost upward. GAO stated the agency’s error regarding the labor cost was not 
nearly as critical as their failure to inform the offeror during discussions. GAO’s holding 
on this point was founded in the FAR requirement (pre and post-Part 15 rewrite) for 
agencies to ensure that discussions (when held) are meaningful. [Ref. 13: 15.601] For 
SOCOM’ s discussions to be meaningful, they would have had to inform Boeing of the 
aspects of their proposal that had to be addressed in order for the offeror to have a 


reasonable chance to obtain award. SOCOM had to discuss the perceived mistake in 
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labor cost. The implication for contracting officers is that they must ensure discussions 
convey any proposal shortcomings for the discussions to be meaningful. [Ref. 24] 

Sylvest, like Boeing shows that an action that indirectly undermines the total cost 
or price comparison can result in a sustained protest. The agency (TVA) failed to 
compare adequately proposal prices by awarding the contract to an offeror who had made 
a necessary hard disk modification (memory increase) an option ın their proposal. GAO 
held that the agency did not consider the optional price increase when they compared 
proposal prices resulting in a lower proposed price by the awardee. In other words, TVA 
did not compare the awardee’s total price (hard disk plus increased memory) to the 
protester’s total price. GAO”s holding looked to the FAR requirement for acceptable 
proposals to be responsive. [Ref. 13: 9.104] The awardee’s proposal was not responsive. 
The implication for contracting officers is that they must be wary of proposals that vary 
from solicitations. If the variance is material, the proposal is unacceptable. [Ref. 26] 

2; Thoroughly Conduct Cost/Price Realism Analyses When Required 

Failure to conduct cost/price realism analyses undermines price evaluation. 
Barents demonstrates that improper cost realism can result in sustainment. The 
solicitation required that all costs be included into fixed multipliers so that the agency 
could evaluate an averaged burdened daily rate in pursuit of best value. The agency’s 
(AID) problem stemmed from an awardee that placed a contingency on their price 


multipliers. The multiplier rate was contingent on the availability of certain property and 
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facilities. The RFP stated that multipliers could have been adjusted through a cost 
realism analysis to evaluate the proposals fairly. A proper cost realism analysis would 
have uncovered the discrepancy (contingency) in the awardee’s proposal. GAO held that 
the cost realism analysis was inadequate. GAO’s holding was founded in the requirement 
that a contract may not materially vary from a proposal. The Comptroller General 
determined that the contract did materially vary from the offer, and the cost realism 
analysis did not shed light on the unacceptable contingency levied by the awardee. The 
implication for contracting officers is that they must make use of tools such as cost/price 
realism analyses. These tools guard against proposals that materially differ from 
solicitations which often leads to contracts that materially differ from the proposals as in 
this case. [Ref. 23] 

Geo showed how the agency’s failure to conduct both cost and price realism 
analyses resulted in a sustained protest. The awardee’s BAFO increase in sample task 
costs, combined with a 25 percent decrease in overall proposal costs, should have drawn a 
cost and price reevaluation by the agency. It did not. Sample tasks and associated cost 
and price realism analyses were required by a solicitation for this task order requirements 
contract. The analyses of the sample tasks were identified by the RFP as only measures 
of contractor understanding. However, GAO stated that the sample task costs were the 
only reasonable method for cost estimation because the cost to the Government would 


have varied based on the issuance of actual task orders. In addition, GAO found that no 
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price realism analyses had been conducted on any of the offerors’ proposals or BAFOs. 
GAO’s holding arose from a previous protest (Group Technologies Corp., B-240736, 19 
December 1990). The Comptroller General had held that in task order contracts, the use 
of sample tasks permitted agencies to review the approaches and efficiencies of different 
contractors under simulated contract conditions which require the submission of detailed 
cost proposals. [Ref. 25] In Geo, where the solicitation advised that sample tasks would 
undergo cost and price realism analyses, the agency was bound to use the evaluation 
methods identified in the solicitation. The implication for contracting officers lies in the 
need for Federal agencies to use the evaluation tools indicated in their solicitations. 
Failure in this regard provides a strong basis for sustainment. [Ref. 25] 

Arora demonstrates how improper and inaccurate price realism analyses can result 
in sustainment. The Navy failed to account for the protester’s plan to counter wage 
escalation through long-term labor agreements. In addition, the agency erroneously 
assessed the protester’s G&A budget for recruitment (in the event of labor turnover). 
GAO held that the evaluation of the protester’s proposal was unreasonable and 
inconsistent. This holding stemmed from an inherent requirement for the Government to 
provide accurate cost realism analyses when such analyses are outlined in the solicitation. 
The implication for contracting officers is that great care must be taken to ensure accurate 


and reasonable price analyses. [Ref. 32] 
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3. Follow the Solicitation 

As in previous chapters, a Federal agency’s inability or unwillingness to follow 
the terms of solicitations results in sustainment. In Arora, the RFP stated that price 
realism analyses would be conducted within the context of the offerors’ past performance. 
With respect to the protester, the Navy ignored the requirement to analyze price realism 
within past performance because the agency had concerns about the risk of wage 
escalation and labor turnover. GAO held that such concerns were valid; however, the 
agency was still required to adhere to the solicitation by evaluating price realism within 
the context of past performance. The Comptroller General bolstered this decision by 
stating that an agency could not arbitrarily make price realism a separate and 
determinative evaluation factor because the solicitation clearly stated that price 
reasonableness would be evaluate within the past performance factor. The implication 
for the contracting officer is that the solicitation must be followed regardless of the 
unusual aspects of an offerors proposal. Fair and consistent evaluation cannot be 
achieved without adherence to the solicitation. [Ref. 32] 

In Tri-State, the awardee gained a price advantage by inappropriately splitting the 
CLINs and offering both first quantity and next quantity prices. GAO held that this 
irregularity benefited the awardee; therefore, the protester was prejudiced. GAO’s ruling 
stemmed from another previous protest (Multi-Spec Prods. Group Corp., B-245156.2, 11 


February 1992) which stated when an irregularity in an offer results in benefits to an 
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offeror, it must be extended to all offerors. None of the other offerors were extended the 
opportunity to make use of the awardee’s pricing scheme. GAO determined that the 
awardee had an unfair advantage and sustained the protest. The implication for 
contracting officers is that the RFP simply must be followed. In addition, improper 
discussions were held regarding the pricing of quantities beyond Government hazardous 
waste estimates. GAO held that the agency’s perceived clarifications were actually 
discussions because they involved information essential to determine the acceptability of 
a proposal. The rule applied by GAO was that if discussions are held with one offeror, 
they must be held with all offerors in the competitive range. [Ref. 13: 15.601] 
Contracting officers must be cognizant of their exchanges with contractors and ensure 
that when discussions are held, all offerors must be included. 

Contracting officer should take these valuable lessons with them when evaluating 
cost/price. First, ensure total price or cost is evaluated and a comparison is held between 
all offerors. Second, execute price and/or cost realism analyses when required. Finally, 
follow the solicitation. Chapter VII examines protests that were sustained due to 


improper agency evaluation of labor qualifications. 
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VII. LABOR QUALIFICATIONS 
A. INTRODUCTION 

Improper Federal agency evaluation of labor qualifications is cited as a sustaining 
argument in nine percent of the successful protests. Labor qualifications are often 
important elements in a best value tradeoff. Clearly, a special qualification brings value 
to an offeror’s proposal as well as associated costs. These valued qualifications must be 
traded off, and through the contracting officer’s judgment, best value is determined. The 
two protests that were sustained due to improper evaluation of labor qualifications are 
Barents Group, L.L.C. and For Your Information, Inc. This chapter thoroughly describes 
the two sustained protests and provides analysis and conclusions regarding agency 
evaluation of labor qualifications. 

B. PROTESTS 

1. Barents Group, L.L.C. 

Barents Group, L.L.C.; B-276082, B276082.2; Comptroller General of the United 
States; 9 May 1997, (Barents) was discussed in Chapter VI because improper agency 
evaluation of cost/price is a sustaining argument in this protest. Improper agency 
evaluation of labor qualifications is also a sustaining argument. Barents protested the 
award of an Agency for International Development (AID) contract for technical expertise 
in: (1) economic and institutional analysis and (2) private sector development issues. The 


RFP outlined the award of contracts in two functional areas; however, protests were 
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issued regarding only line item 0001—privatization issues. Each proposal was to be 
assigned a score for technical and cost. Technical was weighted 60 percent and cost 40 
percent. Technical factors included the following (25 points each): (1) personnel 
qualifications and experience, (2) quality and responsiveness, (3) ona corporate 
experience, and (4) past performance. AID found Chemonics International, Inc. 
(Chemonics); Booz-Allen £ Hamilton, Inc. (Booz-Allen); Abt Associates, Inc. (Abt); 
Carana Corporation (Carana); and Deloitte Touche Tohmatsu (Deloitte) to be the offerors 
who submitted proposals representing the greatest value to the Government. 

Barents argued that the proposals submitted by Chemonics and Carana did not 
comply with the solicitation. The RFP required the submission of three resumes for 
investment bankers to be assigned to the project. The protester maintained that in both of 
the awardees’ proposals, one of the three resumes did not meet the qualification of an 
investment banker. Barents asserted that investment bankers, by definition, underwrite 
and sell new securities. The protester pointed out that Chemonics and Carana had both 
proposed personnel that were mere consultants and not investment bankers. The protester 
went so far as to cite the Dictionary of Banking (Jerry M. Rosenberg (1983)) which stated 
that an investment bank “serves as an underwriter for new issues of bonds or stocks and 
as part of a syndicate, redistributes the issues to investors.” An investment banker was 
defined as “the middleman between the corporation issuing new securities and the 


public.” [Ref. 23] 
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AID countered by arguing that the RFP did not specify the exact qualifications 
required for the investment bankers. Instead, each offeror was to determine the 
competencies necessary to complete the required work. GAO agreed that the RFP did 
not contain definitions of any of the various required labor categories; however, the 
solicitation did ask for separate sets of resumes for each of the labor categories including: 
corporate financial advisor, commercial banker, and investment banker. GAO held that 
the only reasonable interpretation of the RFP was that the resumes for the various labor 
categories should have represented the differences between the categories. In short, the 
terms of the solicitation were not met because inexperienced consultants did not qualify 
as investment bankers. GAO determined that the combination of the improper 
evaluations of price (Chapter IV) and labor qualifications did prejudice Barents. The 
Comptroller General recommended that AID clarify the RFP, request revised BAFOs, 
and reevaluate. [Ref. 23] 

2. For Your Information, Inc. 

For Your Information, Inc., B-278352, Comptroller General of the United States, 
15 December 1997, (FYD is another protest where improper agency evaluation of 
personnel qualifications resulted in sustainment. FYI protested the U.S. Coast Guard 
(USCG) award of a fixed-price, indefinite delivery/indefinite quantity contract for 


information technology support services. The contract was restricted under the Small 
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Business Administration’s 8(a) program. USCG found RGII Technologies, Inc. (RGIT) 
to be the offeror representing best value to the Government. 

The RFP called for a technical evaluation team (TET) to evaluate the proposals in 
two phases. Under Phase I, the TET would evaluate mini-proposals and past 
performance. Phase II would be evaluated under oral presentation/slides, personnel data 
forms (PDF), and cost/price. The proposals were to be evaluated under the following 
color/adjectival system: blue/superior, green/satisfactory, yellow/marginal, and 
red/unsatisfactory. Fifty-one proposals were received. Offerors receiving blue ratings in 
Phase I were invited to continue to Phase II. Offerors not receiving blue ratings could 
continue if they chose to do so. Both FYI and RGII received green ratings in Phase I and 
elected to continue into Phase II evaluations. Phase II was described to be more important 
than Phase I. The solicitation identified ten Phase II labor categories as key personnel 
positions. Offerors were required by the RFP to provide PDFs (containing qualifications, 
education, and employment history) for 50 percent of the proposed personnel in each 
labor category. 

All BAFO’s received by offerors for Phase II evaluation were determined to have 
overall ratings of green and were found to have reasonable and realistic prices. The factor 
in question in this protest was personnel requirements. Only FYI’s BAFO was found 
satisfactory (green) under personnel requirements. RGII’s BAFO was rated as 


yellow/marginal because PDF’s (for 50 percent of the proposed personnel) were not 


106 


submitted for five of the ten labor categories. The TET recommended FYI for award to 
the source selection official (SSO) even though FYI’s price was higher. The SSO asked 
for justification as to why the contract recommendation was being made on a basis other 
than lowest price. The TET responded stating that it doubted RGII’s ability to obtain the 
qualified personnel because they failed to propose qualified workers even after the matter 
was addressed during discussions. The SSO requested information on RGII’s past 
performance and was told it was superior. The SSO found that RGII’s offer was the best 
value for the Government because the risk associated with the lack of qualified personnel 
was outweighed by: (1) past performance and (2) the fact that PDF risk was to be 
minimized by an RFP provision for USCG approval of all PDFs before granting access to 
the work site. The SSO also determined that any proposal shortcomings regarding 
personnel qualifications would be dealt with under applicable labor substitution clauses 
during contract administration. The SSO stated that the cost differential did not justify 
granting FYI the contract award. RGII was awarded the contract, and FYI protested. 

FYI argued that the awardee’s proposal should have been found unacceptable by 
the agency. GAO concurred pointing to the RFP that stated that the quality and quantity 
of all proposed personnel had to meet the minimum qualifications stated in the 
solicitation. GAO asserted that a proposal that fails to conform to the material terms and 
conditions of a solicitation must be considered unacceptable. The Comptroller General 


stated that there was no question that personnel qualifications were material terms of the 
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RFP and held that RGII’s proposal did not satisfy the personnel qualification 
requirements. GAO also stated that qualification problems could not be handled during 
contract administration because the RFP required the contractor to assign the personnel 
named in the PDFs to the contract. Therefore, the only agency a to approve or 
disapprove proposed individuals was during the evaluation—regardless of any clauses 
attached to the contract by the contracting officer. Thus, the agency’s post-award 
approval/disapproval rights were eliminated by this statement in the proposal. The 
Comptroller General found that FYI had been prejudiced. GAO recommended 
termination of RGII’s contract and award of the contract to FYI. [Ref. 28] 
Es ANALYSIS AND CONCLUSION 

The purpose of this analysis is to provide contracting officers with insight as to 
possible agency miscues when evaluating labor qualifications. 

1. Certainty of Requirements is Critical to Successful Evaluation and 

Award 

Barents is a fascinating protest because the outcome hinged on the definition of a 
professional title. The RFP called for three Investment Banker resumes from each 
offeror. Two awardees provided two sets of legitimate resumes that satisfied the 
professional definition of an investment banker. However, each of the two awardees 
provided resumes for consultants for their last (third out of three) investment bankers. 


The protester deftly utilized a dictionary of banking to argue successfully that the 
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consultants were not actually investment bankers. GAO held that the agency did not 
evaluate the proposals in accordance with the RFP. The question at hand was whether or 
not the Investment Bankers had to have experience commensurate with their title. GAO 
held that experience was required. GAO cited the solicitation’s promise that the agency 
would assess whether “the offer [provided] adequate evidence of appropriate academic 
credentials and depth of experience and professional qualifications.” [Ref. 23] The 
Comptroller General determined that this promise placed the offerors on notice that they 
must include in their proposals three resumes of individuals who had worked as 
investment bankers. The third resumes of the two awardees clearly did not contain 
personnel with experience in investment banking. The implication for contracting 
officers lies in the need to explore fully the requirement for which a solicitation is being 
crafted. The arguments of the agency indicated that it did not need three experienced 
investment bankers to satisfy the customer needs. However, the solicitation called for 
three investment bankers. The agency overburdened the RFP with unnecessary 
requirements and failed to hold two awardees to these requirements; consequently, AID 
was unable to defend two of their contract awards to the satisfaction of GAO. 

2: Follow the Solicitation 

As in previous chapters, a Federal agency’s inability or unwillingness to follow 
the terms of solicitations results in sustainments. FYJ is a situation in which the protester 


was the only offeror to meet the RFP requirement for submission of personnel data files 
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(PDF) for 50 percent of the personnel proposed for each labor category. The technical 
evaluation team (TET) recognized this and recommended that the senior selection official 
(SSO) award the contract to FYI. The SSO was swayed only by the awardee’s lower cost 
and ignored the TET’s recommendation. The SSO believed that the PDF problem could 
be dealt with as a contract administration issue, so the award was made to the lower - 
priced proposal. GAO did not concur and held that the awardee’s proposal did not meet 
the material terms of the solicitation. The holding stemmed from a previous protest 
(National Medical Staffing, Inc.; PRS Consultants, Inc., 69 Comp Gen 500, 502 (1990)) 
where the Comptroller General had held that “a proposal that fails to conform to the 
material terms and conditions of the solicitation should be considered unacceptable and 
may not form the basis for an award.” [Ref. 28] 

GAO also determined that the SSO’s intent to handle the PDF problem during 
contract administration was undercut by a provision in the RFP that permitted agency 
approval/disapproval of personnel only during evaluation. By making the award to 
RGII, the agency accepted the individuals that were proposed in an unacceptable manner 
(lacking PDFs for 50 percent of each labor category). The agency, in effect, waived 
RGII’s personnel qualifications requirement, which resulted in unfair and unequal 
evaluation. This particular holding by GAO arose from a “fundamental principle of 
federal procurement that offerors be treated equally; that is, offerors must be provided 


with a common basis for the preparation of proposals, and award based upon the 
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requirements stated in the solicitation, unless the offerors are notified of changes in (or 
relaxation of) the agency’s stated requirements.” [Ref. 28] The implication for 
contracting officers 1s that the RFP must be followed. All provisions must be observed to 
ensure fairness to the offerors. Any changes to the solicitation must be announced and 
made available to all offerors to preserve fairness. 

Contracting officer should take these two valuable lessons with them to avoid 
sustainable protests when evaluating labor qualifications. First, contracting officers must 
ensure the solicitation is crafted to match the requirement. Second, adherence to the 
solicitation is critical to a fair award. Chapter VIII addresses improper agency pre- and 


post-award changes that resulted in protest sustainments in 1997. 
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VIII. IMPROPER AGENCY PRE- OR POST-AWARD CHANGE 

A. INTRODUCTION 

Improper Federal agency pre- or post-award change is cited as a sustaining 
argument in 14 percent of the successful protests in 1997. Improper changes result in the 
unfair evaluation of offerors’ proposals pursuant to a best value award. A change ofa 
significant nature is normally followed by an amendment to the solicitation (pre-award) 
or modification to the contract (post-award). The three protests that were sustained due 
to improper agency changes are /nternational Data Systems, Inc., Marvin J. Perry & 
Associates, and Symetrics Industries Inc. This chapter discusses the FAR requirements 
regarding pre- and post-award changes, describes the three sustained protests, and 
provides analysis and conclusions. 
B. CHANGES AND THE FAR PART 15 REWRITE 

The fundamental regulations regarding changes in Government requirements and 
changes to solicitations are addressed in the FAR Part 15 rewrite; however, there were no 
substantive changes from the previous version. The basic precepts are: 

1. If, either before or after receipt of proposals, the Government changes its 
requirements or terms and conditions, the contracting officer must amend the 
solicitation. 

2. Amendments issued before the established time and date for receipt of 


proposals must be issued to all parties receiving the solicitation. 
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Amendments issued after the established time and date for receipt of proposals 
must be issued to all offerors that have not been eliminated from the 
competitive range. 
4. Ifa proposal of interest to the Government involves a departure from the 
stated requirements, the contracting officer must amend the solicitation 
without revealing to the other offerors the alternate solution or any other 
information that is entitled to protection. [Ref. 1: 15.206, Ref. 13: 15.606] 
Post-award contract modifications are also addressed in both versions of FAR Part 
15 with no significant changes. The applicable regulation states that any changes to 
existing contracts are required to be in scope (minor and within the context of the original 
contract). [Ref. 13: 43.201] Out-of-scope changes undermine the concept of fairness in 
the previous best value evaluation and contract award. 
GC: PROTESTS 

1: International Data Systems, Inc. 

International Data Systems, Inc., B-277385, Comptroller General ofthe United 
States, 8 October 1997, (IDS) was discussed in Chapter V because it is a EA that was 
sustained due to improper agency evaluation of technical merit/ability. ZDS was also 
sustained because of an improper agency change to the solicitation. International Data 
Systems protested the Department of Interior (DOI) award of a fixed-price, indefinite 


delivery/indefinite quantity contract for personal computers with Pentium Pro processors. 
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The contract time period was to span six months. DOI found Applied Computer 
Technology (ACT) to be the offeror representing the best value to the Government. 

The RFP stated that a Technical Evaluation Panel (TEP) would consider the 
following evaluation criteria: conformance with technical specifications—40 percent, 
past performance—40 percent, and price—20 percent. Forty-two offers were received 
with only seven in the competitive range. In addition to the Pentium Pro processors 
specified in the solicitation, ACT proposed a newer alternative: the Pentium II processor. 
ACT’s proposal urged the agency to consider the Pentium II processor. The reason cited 
by ACT was that the RFP stated that replacement parts and support must be available for 
the lifetime of the system, and Intel (the processor manufacturer) had indicated that the 
Pentium Pro would be phased out by the close of 1997. Intel had also informed ACT 
that the Pentium II was the scheduled replacement for the Pentium Pro. The contracting 
officer considered the alternative Pentium II processor offered by ACT as “state of the 
art.” [Ref. 21] The DOI found ACT’s proposal to be the best value and awarded them 
the contract. In ACT’s contract, the agency added a line item (for the Pentium II 
processor) that did not appear in the original solicitation. IDS protested. 

IDS argued that DOI failed to amend the solicitation properly. The Comptroller 
General agreed. GAO stated that agency requirements changes after issuance of a 
solicitation dictate that the agency must amend the solicitation, notify offerors of the 


changes, and afford them an opportunity to respond. Furthermore, the requirement to 


¡es 


amend the RFP exists even after the submission of BAFOs, up to the time of award. 
GAO determined that while the solicitation outlined a Pentium Pro processor, the agency 
determined during the course of the procurement that the Pentium II best represented the 
customer’s needs and should have been incorporated into the contract. GAO found that 
DOI was required to amend the RFP and permit all offerors an opportunity to submit 
revised proposals. The Comptroller General held that the combination of the improper 
agency evaluation of technical merit/ability (Chapter V) and the failure to amend the 
solicitation prejudiced IDS. The protest was sustained. Since the computers were already 
delivered and accepted, GAO recommended reimbursement of proposal and protest costs 
to IDS. [Ref. 21] 

2: Symetrics Industries, Inc. 

Symetrics Industries, Inc.; B-274246.3, B-274246.4, B-274246.5; Comptroller 
General of the United States; 20 August 1997, (Symetrics) is a protest where a significant 
quantity change resulted in sustainment. Symetrics Industries protested the U.S. Air 
Force (USAF) award of a fixed-price, indefinite delivery/indefinite quantity contract for 
Lots IV through VII of the AN/ALE-47 Countermeasures Dispenser System (CMDS). 
The CMDS was an electronic warfare system used by the Army, Navy, and Air Force to 
guard aircraft from hostile missile attacks. The contract was to span four years. USAF 
determined Tracor, Inc. (Tracor) to be the offeror whose proposal represented best value 


to the Government. 
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The RFP outlined the best value proposal to be the one offering the best 
combination of utility, technical quality, business aspects, risks, and price. The 
evaluation factors were identified as (descending order of importance): (1) technical, (2) 
schedule, (3) cost/price (most probable life cycle cost), and (4) management. Tracor had 
been the Government’s sole source for CMDS for the previous eight years. Tracor 
demonstrated in its proposal that it was the only offeror who could best produce the high 
quantity of CMDS units (Lots IV through VII) as well as offer an accelerated delivery 
schedule. USAF awarded to Tracor. Symetrics Industries protested. 

The protester argued that the evaluation of the proposals was unreasonable 
because it was based on an erroneous quantity estimate. The protester maintained that the 
requirement for sequencers in Lot IV (3,219 of the total of all lots combined (3,775)) was 
so unusually high that only Tracor was able to take the extraordinary steps to meet the 
agency's need. The protester did not dispute the original evaluation and contract award. 
However, the agency canceled Lot IV due to lack of funding one week before their 
request for BAFOs and failed to amend the solicitation. Symetrics asserted that many of 
the weakness assessed against its proposal were related to the high number of sequencers 
required in Lot IV. Because the original solicitation no longer reflected an accurate 
quantity estimate, Symetrics asserted that the best value tradeoff in the final evaluation 


lacked a reasonable basis. 
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The agency countered stating that (1) the funding change did not diminish the Air 
Force’s need for the sequencers, (2) funding could have been provided before the end of 
the lot ordering period, and (3) the protester was not prejudiced because the decision not 
to amend the solicitation applied to all offerors equally. GAO stated wise the lot 
cancellation represented a significant change to the Government’s requirement. The 
evaluation record showed that only Tracor was able to avoid schedule risk and meet the 
agency’s need regarding Lot IV. Lots V through VII were much more manageable 
because they totaled only 491, 315, and 503 respectively. The Comptroller General 
stated that the record did not indicate a schedule risk problem (for Lots V through VID 
for the protester in light of the reduced requirements. Furthermore, the prospect of future 
funding (that followed the cancellation) did not impact the agency’s duty to amend the 
solicitation. As aresult, GAO held that Symetrics had been prejudiced and sustained the 
protest. The Comptroller General recommended reopening of the competition, issuance 
of an amended solicitation, and reevaluation. [Ref. 30] 

3: Marvin J. Perry & Associates 

Marvin J. Perry & Associates; B-277684, B277685; Comptroller General of the 
United States; 4 November 1997, (MJP) is protest in which a post-award change 
undermined the best value determination and resulted in sustainment. Marvin J. Perry & 
Associates protested the Department of the Navy award of a contract for 563, ten-piece 


sets of red oak sleeping furniture for a bachelor enlisted quarters at the Naval Training 
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Center, Great Lakes, Illinois. Nine of the ten types of furniture were available from the 
Federal Supply Schedule (FSS). The FSS program is a General Services Administration 
(GSA) program that provides agencies with a simplified process for purchasing 
commonly used supplies and services at prices associated with volume buying. The tenth 
type of furniture (footboards for beds) was solicited in conjunction with the items that 
were listed on the FSS. The Navy determined that DCI’s proposal represented the best 
value to the Government. 

Before the procurement, the Navy contacted an architectural firm to design the 
interior to the enlisted quarters. The firm recommended MJP as the company holding the 
FSS contract offering the lowest priced red oak furniture that met the functional aesthetic 
and quality needs of the service. The Navy solicited quotes from other FSS vendors of 
similar products to ensure that MJP’s products represented the best value to the 
Government. The Navy issued a Request for Quotations (RFQ) for both the nine types of 
furniture available from the FSS as well as the last remaining type (footboards) not listed 
on the schedule. DCI proposed the lowest quote for the nine items at $ 557,127 while 
MJP submitted the next lowest price of $ 572,709. However, MJP offered the lowest 
price of $ 18,016 for the footboards, and DCI quoted the next lowest price of $ 19,789. 
The Navy awarded the contracts to the offeror with the lowest total price—DCI. One 
month before the delivery date, DCI informed the Navy that their supplier had mistakenly 


delivered ash wood instead of the red oak. DCI proposed that the agency either accept 
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the furniture in ash or postpone the delivery date. After reviewing DCI samples of ash 
wood stained to match the appearance of red oak, the Navy decided that the ash furniture 
was an acceptable substitute. The Navy accepted delivery of the ash furniture 
approximately one month later. Upon learning of the wood substitution, MJP protested. 

MJP argued that changing the wood from red oak to ash was unfair to other 
offerors. MJP maintained that they based their quotes on the prospect of providing 
furniture made from the superior grade of wood (red oak) and that they had not been 
given the opportunity to offer quotations using the lower quality wood (ash). MJP 
pointed out that authorization to use ash would have reduced their raw wood costs by 40 
percent and their overall bid price by over $ 47,000. The protester bolstered its assertion 
with copies of the Weekly Hardwood Review that showed that ash orders at the time sold 
for $ 525 per board foot while red oak sold for $ 855 per board foot. Finally, the protester 
stated that the lack of prior agency authorization to change the wood type should have 
caused the Navy to reject DCI’s furniture and make the award to the next lowest quoter-- 
MJP. 

The agency first argued that the protest addressed a contract administration issue 
and was not under the purview of GAO. The Comptroller General stated the Navy’s 
assertion was generally true; however, GAO asserted that out-of-scope modifications to 
existing contracts change the very nature of the original contract award. For this reason, 


the Comptroller General considered the protest. The Navy also stated that local 
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lumberyards had informed them that the scarce nature of ash resulted in only a one to five 
percent reduction in price from red oak. GAO was not swayed and stated that the Navy 
had not presented a basis for the Comptroller General to question the protester’s 
documented contention that substantial savings was to be had through the wood 
substitution. GAO indicated that the best value determination was undermined and stated 
that the “original purpose of the orders was so substantially changed by the modification 
that the original orders and the modified orders are essentially different...By not issuing 
revised RFQs or checking other vendors’ prices for ash furniture, the agency did not 
ensure that it received the best price for the ash furniture, which was the stated purpose 
for issuing the RFPs in the first place.” Because the furniture had already been delivered, 
the Comptroller General recommended that the protester be reimbursed for its protest 
costs. [Ref. 29] 
D. ANALYSIS AND CONCLUSIONS 

The purpose of this analysis is to provide contracting officers with insight as to 
possible agency pitfalls when evaluating labor qualifications. 

l. Contract Awards Cannot be Made to Offerors Proposing Departures 

from Stated Requirements without First Amending Solicitations 

IDS is a protest in which the agency discovered, by virtue of a proposal, that they 

would have been better served with a more modern computer product. The awardee 


(ACT) urged the use of Pentium II processors rather than the Pentium Pro processors 
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solicited for in the RFP. The agency (DOI) issued the contract to ACT and deviated from 
the solicitation by adding a new line item to procure the newer Pentium II processors. 
GAO held that to the extent that DOI’s needs changed to include the Pentium II 
processors, the agency was required to issue an amendment permitting all offerors a 
chance to submit revised proposals on a common basis. The holding was founded in both 
the FAR (15.606—pre-rewnite) and a previous protest (Symetrics—discussed in this 
chapter). Contracting officers must understand that when a proposal brings to light a 
different need for the Government, the need for an amended solicitation also emerges. 
[Ref. 21] 

2 Significant Quantity Changes are Also Changes to Government 

Requirements and Require Amendments to Solicitations 

Symetrics shows contracting officers that quantity changes can sometimes drive a 
need for amendments to solicitations. The USAF wanted to procure several lots of 
electronic warfare sequencers. One week before the receipt of BAFOs, the program 
manager canceled Lot IV (3,219 sequencers) leaving only Lots V though VII (cumulative 
total of 1,309). The agency failed to amend the solicitation and awarded the contract to 
Tracor (previous sole source provider), largely because they had the ability to handle the 
high volume and fast delivery schedule required for Lot IV. The agency maintained that 
the requirement was still valid and that funding could have materialized. GAO found that 


the quantity change was material and sustained the protest. The holding was founded in 
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the FAR (15.606—pre-rewrite) stating that where an agency’s requirements change after 
a solicitation has been issued, it must issue an amendment to notify offerors of the 
changes. GAO continued by stating that one circumstance where an amendment would 
be required is a significant change in the Government’s estimate of the quantity it expects 
to order. Additionally, the Comptroller General considered the funding issue. GAO 
determined that when an agency experiences funding reductions, the agency is required to 
amend the solicitation to reflect the best estimate of the purchase amount. GAO cited a 
previous protest in support of these holdings (Management Systems Designers, Inc., B- 
244383.8, Comptroller General of the United States, 8 June 1992). Contracting officers 
must be cognizant of funding shortfalls as well as quantity changes and ensure amended 
solicitations are generated when significant changes occur. [Ref. 30] 

3: Out-of-Scope Contract Modifications May Result in Sustained 

Protests 

MJP is a prime example of Federal agency failure to recognize an out of scope 
modification. The Navy sought to procure red oak furniture for a bachelor enlisted 
quarters. After awarding to the lowest bidder, the agency approved a change to in wood 
type because the awardee’s supplier had mistakenly delivered ash. GAO determined that 
the original purpose of the order was substantially changed, so the original order and 
modified order were essentially different. There are several bases cited by GAO in this 


holding. First, the out-of-scope modification took a perceived contract administration 
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issue and converted it to a protest issue. GAO stated protests alleging out-of-scope 
modifications will be heard by the Comptroller General because such actions change the 
nature of the original contract award. GAO cited a previous protest in its explanation 
(Indian Native American Employment and Training Coalition, 64 Comptroller General 
460, 1985). Additionally, GAO stated that although the agency conducted this 
procurement under the FSS program, it chose to solicit quotes from other FSS vendors. 
Having held a competition to ensure that they received the lowest priced item, the Navy 
was obligated to ensure that the competition was conducted in a fair manner. The 
Comptroller General stated that the FSS did not exempt an agency from treating vendors 
with the concern for fair and equitable treatment inherent to any procurements. GAO 
cited several previous protests to support this point (Haworth, Inc; Knoll North 
American, Inc., 73 Comptroller General, 1994; SMS Systems Maintenance Services, Inc., 
B-270816, Comptroller General of the United States, 29 April 1996; and Dictaphone, 
Corp., B-254920.2, Comptroller General of the United States, 7 February 1994). A key 
issue in this protest is fairness. There is an implicit contract between the Government and 
vendors that all offerors will be treated fairly and equitably in the procurement process. 
Post-award, out-of-scope modifications violate this concept and invite GAO to intervene 
if protests are submitted. This is precisely why GAO heard and sustained this protest. 


Contracting officers should take extreme care to ensure modifications are within scope. 
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Failure to do so can bring the validity of a best value evaluation and award into question. 
[Ref. 29] 

Contracting officers can take several lessons from this chapter. First, contract 
awards cannot be made to proposals offering departures from stated requirements without 
first amending solicitations. Second, significant quantity changes are also changes to 
Government requirements and require amendments to solicitations. Finally, out-of-scope 
contract modifications may result in sustained protests. Chapter IX provides conclusions 


to the thesis. 
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IX. FINAL ANALYSIS AND CONCLUSION 


A. INTRODUCTION 


The purpose of this chapter 1s to provide contracting officers with an overall 


picture of the pitfalls that exist during the solicitation and contract award process when 


using best value criteria. First, this chapter addresses these problems within the context 


of the contracting process. Second, the reader will be able to see common threads of 


improper agency performance that resulted in protest sustainments in 1997. Third, the 


tradeoff function ıs analyzed. Fourth, the research questions are formally answered. 


Fifth, the chapter conclusion is provided. Finally, areas for future study are outlined. 


The contracting process consists of six steps as shown below: 


l. 


` 


6. 


Acquisition Planning 
Solicitation 

Source Evaluation 
Negotiation 

Contract Award 


Contract Administration 


The next six sections of this chapter discuss the sustained protests of 1997 within the 


context of the contracting process. 
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B. ACQUISITION PLANNING 

Acquisition planning begins upon receipt of a requirement from the customer. 
"Acquisition planning" means the process by which the efforts of all personnel 
responsible for an acquisition are coordinated and integrated through a comprehensive 
plan for fulfilling the agency need in a timely manner and at a reasonable cost. It includes 
developing the overall strategy for managing the acquisition. [Ref. 31: 7.101] The 
method of contracting and type of contract must be determined. A source selection plan 
and statement of work are formulated. Draft requests for proposals are created and pre- 
solicitation conferences may be held. Finally, funding must be verified. 

1. Uncertainty of Requirement 

Barents demonstrates that a protest sustainment can be rooted in poor acquisition 
planning. The protest was sustained because of improper agency (AID) evaluation of 
labor qualifications. The evaluation of labor qualifications were found to be improper 
because two of the awardees failed to provide three resumes for personnel that met the 
professional definition of an investment banker. GAO held that AID had not evaluated 
the proposals in accordance with the solicitation. However, the source of the problem is 
evident from the agency’s counter argument in this matter. AID maintained that despite 
the RFP, three experienced investment bankers were not necessary to satisfy the needs of 
the customer. [Ref. 23] Nevertheless, the solicitation was clear in its call for the three 


investment bankers. Effective acquisition planning relies on an accurate relation of the 
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customer’s requirement to the contracting officer so that the contracting process may be 
executed properly. In Barents, either the customer misstated its need or the agency 
misinterpreted the requirement. Regardless, the protest was sustained because the RFP 
mandated requirements that were not necessary for the customer’s needs to be fulfilled, 
and as a result, the agency did not evaluate the proposal in accordance with the 
solicitation. This uncertainty of the customer’s requirement was a pivotal breakdown in 
acquisition planning. 
C. SOLICITATION 

Solicitations are used in negotiated acquisitions to communicate Government 
requirements to prospective contractors and to solicit proposals. RFPs for competitive 
acquisitions must describe the Government’ s requirement, anticipated terms and 
conditions that will apply to the contract, information required to be in the offeror’ s 
proposal, and factors and significant sub-factors that will be used to evaluate the proposal 
as well as their relative importance. [Ref. 1: 15.203] The ability to craft an effective 
solicitation is critical to the success of the contracting process because the contracting 
officer must select evaluation criteria to be followed during source evaluation/selection. 
Poorly formulated solicitations invariably manifest themselves in improper agency 
actions (evaluations and source selections) in the later stages of the contracting process. 

HG is an example where contradictory requirements in the solicitation resulted in 


an improper agency evaluation of technical merit. The evaluation of technical merit was 


129 


deemed to be improper because the solicitation had contradictory requirements. The true 
requirement of the agency was adequately represented in a conceptual drawing ofa 
proposed office for the Forest Service. The drawing contained a computer room within a 
telecommunications room. However, the drawing was contradicted by an overriding 
written passage dictating that the computer room must be located away from any 
telecommunications equipment. GAO held that the agency did not follow the 
solicitation; however, the root of the protest stemmed from poorly stated requirements in 
the solicitation. It is clear in HG that the Forrest Service wanted a building with a layout 
that closely resembled the conceptual drawing. Unfortunately, the drawing did not match 
the controlling written provision for distance between the computer room and the 
telecommunications equipment. The agency’s inability to convey their needs in the 
solicitation adequately resulted in sustainment. A sloppily written solicitation is a crucial 
mistake threatening the successful completion of the contracting process. 
D. SOURCE EVALUATION/SELECTION 

The objective of source selection is to select the proposal that represents the best 
value to the Government. The award decision is based on an evaluation of factors and 
significant sub-factors that are tailored to the acquisition and must (1) represent the key 
areas of importance and emphasis to be considered in the source selection decision and 
(2) support meaningful comparison and discrimination between and among competing 


proposals. Proposal evaluation 1s an assessment of the proposal and the offeror’s ability 
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to perform the prospective contract successfully. [Ref. 1: 15.302-15.305] Most of the 
sustaining arguments in 1997 were rooted in poor source evaluation and/or source 
selection. The following paragraphs discuss agency improprieties such as failure to 
follow the solicitation, failure to utilize all pertinent facts, improper cost/price evaluation, 
and improper changes to a solicitation. 

1. Failure to Follow the Solicitation Consistently 

There are five sustained protests where the Federal agency failed to follow the 
solicitation when evaluating proposals. TSJ illustrates improper agency evaluation of 
technical merit resulting in the Air Force’s failure to follow the RFP. The awardee had 
failed to provide a scheduling system as prescribed in the solicitation. The agency 
accepted its promise to provide such a schedule and changed the awardee’s adjectival 
score from the lowest to the highest attainable. The schedule never materialized, yet this 
did not hinder the agency from awarding to OTE. 

In a related protest NavCom demonstrated how an improper agency evaluation of 
past performance can result from failure to follow a solicitation. The RFP placed great 
emphasis on “same or similar” TACAN experience. A peripheral agency goal to expand 
the defense base in the TACAN area gave the contracting officer cause to deviate from 
the evaluation criteria. The agency judged every offeror’s past performance as same or 


similar; however, none of the firms had TACAN experience except the protester. 
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Arora is an example of improper agency evaluation resulting from failure to 
follow a solicitation while conducting analyses of price realism. The analyses of price 
realism were supposed to be conducted within the context of the offerors’ past 
performance evaluations. However, the agency (Navy) was so concerned with Arora 
Group’s unusual method of handling wage escalation, they rejected the protester’s 
proposal based solely on perceived price realism problems. The Navy failed to analyze 
price realism within the context of past performance and created a separate and 
determinative evaluation factor that was not part of the stated evaluation criteria. GAO 
held that the Navy essentially ignored the solicitation requirements, so they sustained the 
protest. 

Tri-State is another example of improper agency evaluation of price resulting 
from failure to follow a solicitation. The awardee gained a price advantage by improperly 
splitting the CLINs and offering both first quantity and next quantity prices. The agency 
allowed this price split even though the solicitation had no provision permitting such an 
action. GAO held that this beneficial pricing strategy was not afforded to all offerors, so 
the contract award was made outside the parameters set forth in the solicitation. 

Finally, FY/ demonstrates that an improper agency evaluation of labor 
qualifications can stem from RFP deviation. The agency technical evaluation team (TET) 
recommended award to the protester. The senior selection official ignored the TET and 


awarded to the lowest cost proposal even though the awardee failed to meet the RFP 
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requirement for submission of personnel data files. As in each of the other four protests, 
GAO held that the agency had not followed or held the awardee to the requirements listed 
in the solicitation. 

Deviation from the RFP is the most common misstep regarding source 
evaluation/selection, and it transcends the various sustainment categories. The five 
protests were sustained for improper evaluation of four separate tradeoff elements 
(agency evaluation of technical merit, past performance, price, and labor qualifications); 
however, the root of each sustainment was the same: failure to follow the solicitation. The 
cause may sometimes be blamed on problems that occurred earlier in the contracting 
process such as an unclear requirement (Barents) or a poorly crafted solicitation (HG). 
However, the research shows that there is more often a problem in source evaluation 
and/or selection. Contracting officers that choose not to adhere to solicitations are easy 
protest victims. Contracting officers that inadvertently stray from the RFP parameters 
fair no better under the review of the Comptroller General. Adherence to the RFP is 
crucial to successful source evaluation/selection and ultimately the successful execution 
of the contracting process. 

2. Failure to Utilize all Relevant Facts 

Agency failure to utilize all facts pertinent to a source evaluation/selection was 
found in three sustainments for improper agency evaluation of past performance and one 


sustainment for improper agency evaluation of technical merit. The past performance 
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sustainments are MECSA, STA Engines, and IBSI. In MECSA, fairness and consistency 
were key to the sustainment. The protester was downgraded for late delivery in an 
otherwise satisfactory performance appraisal. Conversely, the awardee was not 
downgraded for safety violations in a previous performance appraisal because the agency 
had determined that the “overall assessment had been satisfactory”. [Ref. 11] This was 
clearly an inconsistent and unfair evaluation of past performance. PCC chose to ignore 
the past performance problems of the awardee, and awarded the contract to FMS. 

IBSI also involved an agency that knowingly disregarded relevant information 
about a protester. In addition to improperly assigning a good past performance rating to 
the awardee (who had no past performance), an employee at the agency (VA) had 
forgotten to fill out a reference form and reduced the protester’s pool of positive 
references from two to one. The contracting officer had a special relationship with the 
protester. This same contracting officer had written and administered the contract--about 
which the information was misplaced. Still the agency failed to consider the information. 
GAO held that the information was clearly known to the contracting officer and should 
have been included. 

STA Engines involved agency problems of a different nature. The agency 
(USCG) erroneously applied irrelevant past performance data about an affiliate and 
unreasonably penalized the protester. GAO determined that the actions of the protester’s 


affiliate had no bearing on its ability to perform the terms of the anticipated contract. The 
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affiliate’s past performance was stated to be important only if the relationship between 
the offeror and affiliate was such that the actions of one party directly affected the ability 
of the other party to perform. 

Cygnus is an example of improper agency evaluation of technical merit stemming 
from a failure to read the protester’s BAFO thoroughly. The protester had originally 
forgotten to include a statement of potential contract problems and solutions in the 
proposal. The required information was included in the protester’s BAFO, but the agency 
did not factor the information in the final evaluation. GAO held that the relevant 
information should have been evaluated. In all four cases, the Comptroller General 
determined that an improper agency evaluation had occurred because of the agencies’ 
failure to assemble, interpret, and utilize relevant information properly about awardees 
and/or offerors. 

In each of the three past performance improprieties, there 1s one driving theme: 
relevance of information. If past performance information is relevant to an offeror’s 
ability to satisfy the terms of a contract, it must be factored into the source evaluation and 
selection. If it ıs irrelevant, it must be withheld from evaluation. Contracting officers 
must carefully screen and interpret all contractor information to determine if it is relevant 
to the contemplated contract. In Cygnus (improper evaluation of technical merit), there 
was no question of relevance--only a lack of attention to detail by the contracting agency 


when evaluating technical merit. The implication for the contracting officer is that proper 
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use of information is key to fair and impartial source evaluation and selection. It is 
fundamental to the successful completion of the contracting process in pursuit of a best 
value award. 

3. Remember the Importance of Cost/Price 

Cost/Price is the central and most important tradeoff element. Contracting 
officers cannot forget that total cost/price must be evaluated in every negotiated 
procurement. Use of other cost/price related data (e.g. cost msk) may prove to be helpful 
in source evaluation and selection; however, the contracting officer is never relieved of 
the requirement to evaluate and compare total cost/price. Boeing and Sylvest are two 
examples where agency failure to evaluate total cost/price resulted in sustainment. In 
Boeing, the Federal agency (SOCOM) was unfamiliar with the requirement to evaluate 
and compare total cost even though their solicitation clearly and correctly dictated this 
action. Sylvest involved a less obvious miscue. The agency (TVA) was found to have 
evaluated and compared the total prices inadequately because they awarded the contract 
to a firm that had listed a necessary computer hard disk modification as optional. GAO 
held that by not including the price of the option in the price evaluation, TVA failed to 
compare the awardee’s total price to the protester’s total price. Evaluation and 
comparison of offerors’ total cost/price is a fundamental precept in source evaluation and 
selection. A protest that establishes a Federal agency failure of this type is very 


sustainable. 
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Another important cost/price issue is effective evaluation of cost or price realism. 
Barents, Geo, and Arora all demonstrate that agency failure in this regard may result in 
sustainments. In Barents, the agency cost realism analysis was found to be inadequate 
because it failed to uncover an improper contingency placed upon the awardee’s cost 
proposal. In Geo, cost and price realism analyses were required. The agency simply did 
not conduct price realism analyses on any of the proposals or BAFOs. Regarding cost 
realism, an agency misconception about the relationship between sample task costs and 
task order requirements contracts doomed the evaluation. The solicitation identified 
sample task cost analyses as measures of contractor understanding only. GAO 
determined that sample task costs were the only reasonable method for cost estimation in 
such a contract because the actual costs to the Government would certainly have varied 
according to the issuance of task orders. 

Arora shows that an erroneous agency analysis of a protester’s price realism 
results in sustainment. The protester was downgraded because the Navy had concerns 
about its failure to provide for wage escalation in its price proposal. The protester had 
explained that its long-term employment agreements and a budget for recruitment in their 
General and Administrative costs precluded the need for wage escalation provisions. 
GAO agreed and held that the agency’s analysis of price reasonableness unreasonably 
discounted the labor agreements and erroneously undervalued the G&A budget 


(recruitment) by 90 percent. It is very important for contracting officers to maintain an 
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awareness of cost/price issues. Failure to evaluate and compare total cost and improper 
cost/price realism analyses properly was a sustainable argument in five of the 21 
successful protests. As a result, the source evaluation/selection was compromised and the 
contracting process disrupted. 

4. Amend Solicitations to Reflect Changes in Government Requirements 

Improper changes result in an unfair best value evaluation and source selection. 
IDSI and Symetrics are good examples of agency failure to amend a solicitation to reflect 
changes in Government requirements. In ZDSI, the awardee had proposed an alternative 
state-of-the-art computer processor (Pentium II) in addition to the one for which the 
solicitation was issued (Pentium Pro). The resultant contract for delivery of computers 
with the Pentium II processor was a deviation from the stated requirements in the 
solicitation. No amendment to the solicitation had been issued, and the result was a 
protest sustainment. Symetrics demonstrates that a significant quantity change is a 
change to the Government requirement and requires an amendment to the solicitation. 
One week before BAFOs, funding was canceled for the first lot in the contract. This lot 
represented 71 percent of the total contracted quantity. The agency felt an amendment to 
the solicitation was not necessary because of its continued need for the goods and a 
potential for increased funding. GAO stated that potential funding did not change the fact 
that the quantity was significantly changed, and an amendment was required. The 


implication for contracting officers is that significant changes to Government 


requirements (that occur up to the point of award) must be reflected in amendments to 
solicitations. This is to ensure fairness for all offerors. The penalty for award without an 
amendment is an improper source evaluation and selection that cripples the contracting 
process. 

E. NEGOTIATIONS/DISCUSSIONS 

Negotiations are exchanges that occur after the establishment of the competitive 
range. They are undertaken with the intent of allowing offerors to revise their proposals. 
Discussions are negotiations tailored to each offeror’s proposal and if conducted, must be 
held with each offeror in the competitive range. The purpose of discussions is to 
maximize the Government’s ability to obtain the best value based on the requirement and 
the evaluation factors set forth in the solicitation. [Ref. 1: 15.306] Failure to hold 
meaningful discussions was prevalent in several protests in 1997. 

The issue of discussions is key to five of the 21 of the protest sustainments. 
Although this issue transcends the various tradeoff elements, contracting officers 
soliciting for best value awards seemed to have difficulty with discussions pertaining to 
past performance and cost/price in 1997. Three of the protests in question were sustained 
in part because of an agency failure to hold meaningful discussions about past 
performance. Two protest sustainments involved discussions regarding cost/price 


information. 
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Discussions regarding poor past performance reports must be held to give offerors 
an opportunity to account for the situation. AC/ illustrates that even though the FAR is 
clear in this regard, mistakes can still be made. The agency mentioned only one of two 
negative past performance reports in discussions. Therefore, the discussions were not 
meaningful, and the protest was sustained. McHugh shows that the requirement to hold 
discussions applies to internal agency references as well as third party reports. The 
agency maintained that intra-agency reports were not subject to interpretation, so 
discussions were not required. GAO held that the FAR makes no such distinction and 
sustained the protest. STA Engines warns of the danger of blanket use of an affiliate’s 
past performance as an indication of an offeror’s ability to meet the terms of a contract. 
The Federal agency improperly applied the poor past performance of the protester’s 
affiliate. In addition, they failed to inform the protester of its intent to use the affiliate’s 
poor standing as an indication of their ability to perform. Again, a sustainment occurred, 
in part, because the agency failed to hold meaningful discussions about a poor report. 

Regarding cost/price evaluation, 7ri-state demonstrates that contracting officers 
must understand the difference between clarifications and discussions. When discussions 
are held, they must be held with all offerors in the competitive range. The awardee had 
proposed a different (unauthorized) pricing scheme by splitting the CLINs. In addition, 
the contracting officer wanted to clarify whether the next quantity ordered price would 


apply to future quantities. The Comptroller General held that they agency’s perceived 
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clarifications (about the price of follow on quantities) were actually discussions. The fact 
that these discussions were not held with all offerors contributed to the sustainment of 
this protest. In Tri-State, GAO appeared to draw a firm distinction between clarifications 
and discussions. This effort, in the pre-rewrite months of 1997, may have been designed 
to clearly define these concepts before the rewrite introduced the newest form of 
exchange--communications. Boeing is a protest where the agency first failed to 
accurately conduct cost evaluation (labor hours and overhead) and then failed to inform 
the protester during discussions. GAO held in this protest that the errors in cost 
evaluation were not, by themselves, sustainable miscues. However, the evaluation errors 
coupled with the failure of the agency to raise the perceived problems during discussions 
resulted in sustainment. 

Discussions are not always the major flaw in a protest. Sometimes the lack of 
discussions simply accompanies an agency impropriety in evaluation of one of the 
tradeoff elements. The key behind meaningful discussions is that if properly held, 
protests may be prevented. In STA Engines it is quite possible that if the protester had 
been informed of the agency’s application of its affiliate’s past performance, objections 
by the protester could have caused agency to research the issue further and make proper 
source evaluation decisions. Tri-State is another example where discussions may have 
uncovered the awardee’s unfair pricing scheme and caused the agency to extend the 


benefits of the scheme to all offerors. In Boeing, discussions may have resulted in 
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correction of the agency’s errors during cost evaluation. Such actions would have made 
these contract awards valid. The implication for contracting officers is that the FAR Part 
15 rewrite encourages even more exchanges between agencies and offerors than were 
permitted under the former regulation. Contracting officers must make use of the 
increased opportunity for communications, clarifications, and discussions to eliminate 
situations such as ACI, McHugh, and STA Engines where discussions simply were not 
held. However, Tri-State demonstrates that contracting officers must be fully aware of 
the types of exchanges they are utilizing to prevent misapplication of clarifications, 
discussions, or communications. Highly trained contracting officers will be able to 
effectively use the expanded exchanges to ensure fair and impartial source 
evaluation/selection and proper best value award. 
F. CONTRACT AWARD 

Contract award is made to the offeror whose proposal represents the best value to 
the Government. One situation that undermines a best value award is a lack of supporting 
narrative for the award. This problem seems to be rooted in evaluations of technical 
merit. Three GAO protest decisions indicated that a lack of narrative contributed to 
sustainment, and all three of these protests were sustained due to improper agency 
evaluation of technical merit. 

In MECSA, the agency (PCC) provided no justification in the evaluation record 


for overriding the protester’s lower price when making award. Also, the lack of narrative 
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provided no proof that PCC had accounted for a special certification held by one of the 
protester’s subcontractors. Jones demonstrates that numerical score sheets are not 
enough to document the best value tradeoff. The protester successfully argued that the 
agency could not justify its award because the evaluation record consisted only of score 
sheets with terse bullet narrative. In JW, the dearth of narrative provided no explanation 
as to why the Federal agency scored the awardee higher under an evaluation factor when 
the protester appeared to be significantly superior. The only supporting documentation 
consisted of a one paragraph per offeror narrative that failed to explain any of the scoring 
or reasons for award. 

The Government’s previous preoccupation for low cost/price gave rise to a 
situation where thorough narrative was not necessarily critical in support of cost/price 
analyses, matrices, and score sheets. In best value contracting a tradeoff occurs, and price 
is now only one of several significant concerns. It is important to note that narrative is 
critical because the benefit of a tradeoff is not self evident from a numerical or adjectival 
score. The implication for contracting officers is that best value contracting drives a 
higher standard for supporting narrative. Neither GAO nor the FAR Part 15 rewrite 
address the quantity or quality of narrative required to adequately support a best value 
award. In MECSA, Jones, and JW, supporting narrative was either absent or barely 
existent. The best value tradeoff is difficult to express numerically. GAO appears to be 


calling for contracting officers to document and justify their award thoroughly. 
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Supporting comments give the contracting officer a chance to outline the rationale for 
their source selection fully. Strong award decisions are built on a foundation of strong 
evaluation and supporting narrative. Agency awards without such narrative are weak and 
prone to sustainment. | 
G. CONTRACT ADMINISTRATION 

Contract administration is performed by either the PCO, or through delegation, 
by an ACO at a Contract Administration Office (CAO). The overarching theme to 
contract administration is oversight, and protests are often thought to be a past danger. 
However, it is key to understand that protests can still occur through action or inaction 
during contract administration. FYZ illustrates that a contracting officer’s plan to address 
a problem through contract administration may result in an improper contract award anda 
sustained protest. The senior selection official (SSO) believed that the awardee’s failure 
to provide personnel data files (PDF) for 50 percent of the personnel proposed for each 
labor category could be handled through a substitution clause in contract administration. 
As aresult, the SSO’s award was successfully challenged because GAO held that the 
awardee had not met the requirements of the solicitation. 

Clauses allowing the Government to make adjustments in the post-award phase do 
not relieve the contracting officer from making a fair and impartial best value award. 
MJP demonstrates that out-of- scope changes shift the very foundation of a best value 


award and can be the subject of a protest. The agency allowed the awardee to deliver 
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furniture made of ash instead of the much more expensive red oak called for in the 
solicitation. GAO held that the original purpose of the order was substantially changed, 
and the original order and modified order were essentially different. The best value 
award was invalidated. The implication for contracting officers is that out of scope 
changes can convert contract administrative issues into protest issues. GAO will hear 
protests alleging out-of-scope changes and modifications. In this situation, the ten day 
protest window ts essentially waived by GAO. The clock begins with a protesters first 
knowledge about an alleged miscue. A post-award agency impropriety and subsequent 
protest could occur months or years after award. Provided that the protester files the 
protest in a timely manner (after gaining knowledge of an alleged offense), GAO will 
hear the protest. In sum, Federal agencies must understand that protests can be 
promulgated by a contracting officer’s actions during contract administration. Such 
actions can destroy all of the hard work that goes into a solid and proper best value 
award. 
H. TRADEOFFS 

This researcher finds no evidence that the act of making tradeoffs was problematic 
for contracting officers in 1997. Because tradeoffs are the very heart of a best value 
award, one may have assumed that problems with best value awards could be traced to 
improper tradeoffs by a Federal agency. The evidence suggests otherwise. Improper 


agency tradeoff was alleged in 40 of the 108 best value protests in 1997. Interestingly, 
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allegations of improper agency tradeoff were never sustained. The implication is that the 
Comptroller General supports the contracting officers’ judgment regarding best value 
tradeoffs provided that they (1) evaluate the various tradeoff elements (cost/price, 
technical merit, past performance, labor qualifications) properly, (2) award the contract 
without making improper pre-award changes to the solicitation, and (3) do not make out- 
of-scope post-award changes. 

I. RESEARCH QUESTIONS 

l; Primary Research Question 
To what extent does the exercise of Federal agency contracting officer judgment, in best 
value source selection, result in a higher rate of GAO protest sustainment than other 
methods of source selection? 

As stated in Chapter III, the research evidence shows that 21 of the 108 best value 
protests were sustained rendering a sustainment rate of 19.44 percent. GAO’s 
sustainment rate for all protests in 1997 was 12 percent. 

er Subsidiary Research Questions 
What is Best Value, the genesis of the concept, the evolution of this type of contracting, 
current implementation, and the regulations that apply? 

As stated in Chapter II, there is no definitive starting point for the concept; 
however, businesses have historically sought to purchase goods and services at a value to 


facilitate their pursuit of profit. The Government focus on best value began with the 
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1986 Packard Commission recommending the use of commercial practices. DoD 
officially recognized the best value concept in a study entitled Best Value Evaluation 
Process. In the 1990’s, acquisition reform measures such as FASA and the efforts of the 
Clinton Administration placed best value squarely at the center of Government 
contracting. These efforts culminated in the rewrite of the FAR Part 15 in which best 
value was defined and recognized as the goal of every Government procurement. 

In sum, best value is the expected outcome of any acquisition that ensures the 
customer’s needs are met in the most effective, economical, and timely manner. It is the 
result of the combination of: the unique circumstances of each acquisition; the acquisition 
strategy; choice of contracting method; and the award decision. Best value is the goal of 
sealed bidding, simplified acquisition, commercial item acquisition, negotiated 
acquisition, and any other specialized acquisition method or combination of methods. 
Through the best value continuum the Government always seeks to obtain the best value 
in negotiated acquisitions using any one or a combination of source selection approaches, 
and that acquisition should be tailored to the requirement. At one end of this continuum 
is the lowest priced-technically acceptable strategy and at the other end is a process by 
which elements of a proposed solution can be traded off against each other to determine 
the solution that provides the Government with the overall best value. All such tradeoffs 
must be conducted according to the source selection factors and sub-factors identified in 


the solicitation. [Ref. 3: p. 1] 
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Can the reasons for the sustainments be categorized? If so, what are the categories? 

As stated in Chapter 111, the researcher has categorized the reasons sustainment by 
tradeoff element. The categories are identified by protester allegations of Government 
impropriety in their agencies’ evaluation of the following tradeoff A cost/price, 
past performance, technical merit, and labor qualifications. One separate category has 
been created to accommodate improper agency pre- and post-award changes that 
invalidated best value awards. 

Are there measures that a contracting officer can take to preclude protests regarding 
Best Value awards? 

Chapters IV through IX have identified lessons contracting officers can learn from 
the 1997 sustainments to avoid improprieties that could result in sustained protests. 
These lessons include: 

e The need for contracting officer certainty of the requirement during 

acquisition planning. 

e The importance of a well-crafted solicitation. 

e The critical requirement to follow the solicitation during source evaluation 

and selection. 

e The need to use all relevant facts during source evaluation and selection. 

e The importance of cost/price evaluations and cost/price realism analyses 


during source evaluation and selection. 
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e The dangers of failing to amend a solicitation when changes in Government 
requirements occur during source evaluation and selection. 

e The need for supporting narrative to support the best value award. 

e The fact that post-award out-of-scope changes can result in sustainable 
protests. 

e The understanding that a contracting officer’s intent to fix a problem with a 
potential awardee’s proposal during contract administration must not unfairly 
prejudice the other offerors. 

How may contracting officers be better trained regarding best value awards? 
Contracting officer training should focus on the traditional importance of 
cost/price as a tradeoff element; however, increased emphasis should be applied to 
evaluation strategies and techniques for the other tradeoff elements (past performance, 
technical ment, and labor qualifications). Furthermore, contracting officer training must 
enhance awareness that pre- and post-award changes could invalidate an otherwise proper 
best value source selection and result in a protest and sustainment. 
J. CONCLUSION 

It is in the basics that contracting officers failed in 1997. Evaluations of the 
elements of tradeoff were often improper because of unsure requirements, lack of 
discussions, failure to use relevant facts, failure to consider total price/cost, etc. It is 


important to note that the FAR Part 15 rewrite in the spirit of acquisition reform 1s 
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designed to create a smoother path to contract award through innovative practices such as 
best value as well as improving quality and preserving fairness. However, best value in 
some ways creates a higher standard for the contracting officer. Before, the focus was 
simply cost/price or related factors. Now the contracting officer must deftly evaluate as 
many as four tradeoff elements including cost/price. It is possible that the requirement 
for thorough evaluation of multiple tradeoff elements could actually increase the level of 
the contracting officer’s workload since the days of the dominant focus on cost/price. In 
sum, best value does not grant the contracting officer a license to make arbitrary contract 
awards. GAO seems to be consistent in their approach to best value contracting. 
Contracting officer judgment is still well supported by GAO, but the Comptroller 
General’s continuing focus on fairness and impartiality is the cornerstone for the 21 
sustainments. Contracting officers must be mindful of this fact as they increase their use 
of best value source selection. 
K. AREAS FOR FURTHER RESEARCH 
The following areas for future research regarding best value are: 
1. Conduct follow-on analyses of best value protests for years beyond 1997. 
2. Develop a model to assist contracting officers when engaged in best value 


contracting. 
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Conduct an in-depth analysis of a specific tradeoff element (cost/price, technical 
merit, past performance, or labor qualifications) as it relates to best value 
contracting. 

Conduct an in-depth analysis of out-of-scope changes to contracts as they relate to 
best value contracting. 


Conduct a joint analysis of best value contracting with GAO. 


15] 





APPENDIX 


PROTESTS OF 1997 INVOLVING BEST VALUE CRITERIA 


Symbols: S-Successful Best Value Protest; U-Unsuccessful Best Value Protest; 


NC-Did Not Meet Criteria as Best Value 


Symbol Protest Name 


U 


NC 


NC 


NC 


U 


NC 


U 


NC 


S 


NC 


AAC Associates, Inc., 

Aalco Forwarding, Inc. 

ACS Systems & Engineering, Inc., 
Advanced Designs Corporation 

Alcan Environmental, Inc., 

All State Boiler Work, Inc., 

Alliant Techsystems, Inc., 

Allied Signal, Inc., Electronic Systems, 
American Combustion Industries, Inc., 
American Marketing Associates, Inc., 
American Native Medical Trans. L.L.C., 
Ameriko, Inc., 

Analytical & Research Technology, Inc., 
Ares Corporation, 


Arora Group, Inc., 


155 


Decision 


B-274928 


B-277241.12/.13 


B-275439.3 


B-275928.2/.3 


B-275859.2 


B-277362 


B-276162/.2/.3 


B-275032/.2 


B-275057.2 


B-274454.4 


B-276873 


B-277068 


B-276064 


B-275321/.2 


B-277674 


13-Jan-97 
29-Dec-97 
31-Mar-97 
21-Apr-97 
11-Apr-97 
3-Oct-97 
2-May-97 
17-Jan-97 
5-Mar-97 
14-May-97 
5-Aug-97 
29-Aug-97 
7-May-97 
7-Feb-97 


10-Nov-97 


NC 


NC 


NC 


NC 


NC 


NC 


NC 


Assets Recovery Systems, Inc., 
Atlantic Scientific & Technology Corp. 
ATLIS Federal Services, Inc., 

AV IATE EC. 

Barents Group, L.L.C., 

BE, Inc.; PAI Corporation 

Best Foam Fabricators, Inc., 

Boeing Sikorsky Aircraft Support, 
Booz-Allen & Hamilton, Inc., 

Brisk Waterproofing Company, Inc., 
Bristol-Myers Squibb Company, 
California Environmental Engineering, 
Canadian Commercial Corporation 
Charles E. Smith Companies 

Chek F. Tan & Company, 

CHI Fabrication Services, 

Cincom Systems, Inc., 

CitiWest Properties, Inc., 


Command Enterprises, Inc., 


Commonwealth Industrial Specialties, Inc., 


154 


B-275332 


B-276334.2 


B-275065.2/.3 


B-275058.6/.7 


B-276082/.2 


B-277978/.2 


B-275436 


B-277263.2/.3 


B-275934.2 


B-276247 


B-275277 


B-274807/.2 


B-276945 


B-277391 


B-277163 


B-275079 


B-275055/.2 


B-274689.4 


B-277042 


B-277833 


10-Feb-97 
27-Oct-97 
12-Feb-97 
14-Apr-97 
9-May-97 
16-Dec-97 
20-Feb-97 
29-Sep-97 
29-May-97 
27-May-97 
5-Feb-97 
3-Jan-97 
31-Jul-97 
25-Sep-97 
8-Sep-97 
22-Jan-97 
21-Jan-97 
26-Nov-97 
21-Aug-97 


25-Nov-97 


NC 


Computer Systems Development Corp. 
Consolidated Engineering Services, Inc., 
Cordev, Inc., 

Creative Apparel Associates, 

Crown Clothing Corporation, 

Cubic Applications, Inc., 

Cygnus Corporation, 

DIGICON Corporation, 

Doss Aviation, Inc.; Dominion Av, Inc. 
Dynalantic Corporation, 

DynaLantic Corporation, 

Dynamic Resources, Inc., 

Eagle Design & Management, Inc., 
EastCo Building Services, Inc., 

EBA Engineering, Inc. , 

ECC International Corporation, 

ECG, Inc., 

Ecology and Environment, Inc., 
ENDMARK Corporation, 


Environmental Affairs Management, Inc., 
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B-275356 


B-277273 


B-275596 


B-275139 


B-277505.2 


B-274768/.2/.3 


B-275181 


B-275060/.2 


B-275419/.2/.3 


B-274944.2 


B-274944.5 


B-277213/.2 


B-275062 


B-275334/.2 


B-275818 


B-277422/.2 


B-277738 


B-277061.2 


B-278139 


B-277270 


11-Feb-97 
24-Sep-97 
7-Mar-97 
24-Jan-97 
31-Oct-97 
2-Jan-97 
29-Jan-97 
21-Jan-97 
20-Feb-97 
25-Feb-97 
25-Aug-97 
16-Sep-97 
21-Jan-97 
10-Feb-97 
31-Mar-97 
14-Oct-97 
20-Oct-97 
5-Sep-97 
31-Dec-97 


23-Sep-97 


NC 


NC 


NC 


Environmental Chemical Corporation, 


Environmental Training&Consltg Intl, Inc., 


Epoch Engineering, Inc., 

Federal Computer International Corp. 
Fluid Power International, Inc., 

Food Services of America, 

For Your Information, Inc., 
GEC-Marconi Electronic Systems Corp. 
General Physics Federal Systems, Inc., 
Geo-Centers, Inc., 

Global Associates, Ltd., 

GTE Hawaiian Telephone Company, Inc., 
H.F. Henderson Industries, 

HG Properties A, L.P., 

HSG-Holzmann Technischer Services 
HSG-SKE, 

IGIT, Inc., 

Ingalls Shipbuilding, Inc., 

International Business Systems, Inc., 


International Data Products, Inc. 
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B-275819 


B-278185 


B-276634 


B-276885 


B-278112 


B-276860 


B-278352 


B-276186/.2 


B-275934 


B-276033 


B-275534 


B-276487.2 


B-275017 


B-277572/.2/.3 


B-274992.2/.2 


B-274769/.3 


B-275299.2 


B-275830/.2/.3 


B-275554 


B-275480.2/.3/.4 


]-Apr-97 
5-Dec-97 
7-Jul-97 
29-Jul-97 
10-Dec-97 
28-Jul-97 
15-Dec-97 
21-May-97 
21-Apr-97 
5-May-97 
3-Mar-97 
30-Jun-97 
17-Jan-97 
29-Oct-97 
26-Feb-97 
6-Jan-97 
23-Jun-97 
7-Apr-97 
3-Mar-97 


3-Apr-97 


NC 


NC 


NC 


NC 


NC 


International Data Systems, Inc., 


J.A. Jones Management Services, Inc., 


Jack Faucett Associates, 

JEOL USA, Inc., 

Jet Investments, Inc., 

JW Associates Inc., 

JW Associates Inc., 

L.A. Systems, Inc., 

Liebert Federal Systems, Inc., 
Litton Systems, Inc., Amecom Div. 
Logicon RDA, 

MAC's General Contractor, 

Main Building Maint., Inc--Costs, 
Management Resources, Inc., 
Marvin J. Perry & Associates, 
Matrix International Logistics, Inc., 
MCA Research Corporation, 
McHugh/Calumet, a Joint Venture, 
Mechanical Contractors, S.A. 


Minotaur Engineering, 


7 


B-277385 


B-276864 


B-277555 


B-277160 


B-276215/.2 


B-275209 


B-275209.3 


B-276349 


B-274823 


B-275807.2 


B-276240/.2/.3 


B-276755 


B-260945.6 


B-276424/.2 


B-277684/5 


B-277208/.2 


B-276865 


B-276472 


B-277916 


B-276843 


8-Oct-97 
24-Jul-97 
12-Sep-97 
2-Jul-97 
22-May-97 
30-Jan-97 
22-Jul-97 
9-Jun-97 
8-Jan-97 
16-Apr-97 
23-May-97 
24-Jul-97 
15-Dec-97 
13-Jun-97 
4-Nov-97 
15-Sep-97 
29-Jul-97 
23-Jun-97 
27-Oct-97 


22-May-97 


NC 


NC 


NC 


MiTech, Inc., 

Motorola, Inc., 

National Office Systems, Inc., 
Navales Enterprises, Inc. 

NavCom Defense Electronics, Inc., 
Nomura Enterprise, Inc., 
Northwest Management, Inc., 
Ouachita Mowing, Inc., 

Pearl Properties, Inc., 

Pemco Aeroplex, Inc--Recon.& Costs, 
Polar Power, Inc., 

PRB Associates, Inc., 

PRC, Inc., 

PRC, Inc.--Reconsideration, 
Precision Echo, Inc., 

Quality Elevator Company, Inc., 
Resource Management Intl, Inc. 
Richard M. Milburn High School, 
Rockville Mailing Service, Inc., 


Rotair Industries, Inc., 
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B-275078 


B-277862 


B-274785 


B-276122 


B-276163 


B-277768 


B-277503 


B-276075/.2 


B-277250.2 


B-275587.5/.6 


B-274846.2/.3 


B-277994/.2 


B-274698.2/.3 


B-274698.4 


B-276740/.2 


B-276750 


B-278108 


B-277018 


B-275073/.2 


B-276435.2 


23-Jan-97 
3-Dec-97 
6-Jan-97 
13-May-97 
19-May-97 
19-Nov-97 
20-Oct-97 
8-May-97 
18-Sep-97 
14-Oct-97 
14-Jan-97 
18-Dec-97 
23-Jan-97 
10-Jul-97 
23-Jul-97 
23-Jul-97 
22-Dec-97 
19-Aug-97 
23-Jan-97 


15-Jul-97 


NC 


NC 


Rotary Furnishing Company, 
Roxco, Ltd., 

Science and Engineering Svcs. Inc., 
SEEMA, Inc., 

Shel-Ken Properties, Inc., 

Siebe Environmental Controls, 

ST Aerospace Engines Pte. Ltd., 
ST Aerospace Engines Pte. Ltd., 
Suddath Van Lines, Inc. 


SWR, Inc., 


Sylvest Management Systems Corp. 


Symetrics Industries, Inc., 


T. Head and Company, Inc., 


Technology Services International, Inc., 


Tecom, Inc., 

Telford Aviation, Inc., 
Terex Cranes, Inc., 
Texstar, Inc., 

The Cube Corporation, 


The Leader Mortgage Company, 
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B-277704 


B-277545 


B-276620 


B-277988 


B-277250 


B-275999.2 


B-275725 


B-275725.3 


B-274285.2/.3 


B-276878 


B-275995/2 


B-274246.3/.4/.5 


B-275783 


B-276506 


B-275518.2 


B-275896 


B-276380 


B-275961/80 


B-277353 


B-274110.2 


13-Nov-97 
27-Oct-97 
3-Jul-97 
16-Dec-97 
18-Sep-97 
12-Feb-97 
19-Mar-97 
17-Oct-97 
19-May-97 
29-Jul-97 
21-Apr-97 
20-Aug-97 
27-Mar-97 
21-May-97 
21-May-97 
16-Apr-97 
10-Jun-97 
22-Apr-97 
2-Oct-97 


19-May-97 


NC 


Theisinger und Probst 

TMI Services, Inc., 

Trend Western Technical Corp. 
Tri-State Government Services, Inc., 
UNICCO Government Services, Inc., 
United Ammunition Container, Inc., 
United Terex, Inc., 

USA Electronics, 

ViON Corporation, 

Wastren, Inc., 


Wind Gap Knitwear, Inc., 
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B-275756 


B-276624.2 


B-275395.2 


B-277315.2 


B-277658 


B-275213 


B-275962.2 


B-275389 


B-275063/.2 


B-276093 


B-276669 


25-Mar-97 
9-Jul-97 
2-Apr-97 
15-Oct-97 
77-Nov-97 
30-Jan-97 
30-May-97 
14-Feb-97 
4-Feb-97 
12-May-97 


10-Jul-97 


10. 


11. 
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